Electronic oven with infrared evaluative control

ABSTRACT

A disclosed computer-implemented method for heating an item in a chamber of an electronic oven towards a target state includes heating the item with a set of applications of energy to the chamber while the electronic oven is in a respective set of configurations. The set of applications of energy and respective set of configurations define a respective set of variable distributions of energy in the chamber. The method also includes sensing sensor data that defines a respective set of responses by the item to the set of applications of energy. The method also includes generating a plan to heat the item in the chamber. The plan is generated by a control system of the electronic oven and uses the sensor data.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.15/467,975, filed Mar. 23, 2017, which claims the benefit of U.S.Provisional Application No. 62/315,175, filed Mar. 30, 2016, U.S.Provisional Application No. 62/445,628 filed Jan. 12, 2017, U.S.Provisional Application No. 62/349,367, filed Jun. 13, 2016, and U.S.Provisional Application No. 62/434,179, filed Dec. 14, 2016, all ofwhich are incorporated by reference herein in their entirety for allpurposes.

BACKGROUND OF THE INVENTION

Electronic ovens heat items within a chamber by bombarding them withelectromagnetic radiation. In the case of microwave ovens, the radiationmost often takes the form of microwaves at a frequency of either 2.45GHz or 915 MHz. The wavelength of these forms of radiation are 12 cm and32.8 cm respectively. The waves within the microwave oven reflect withinthe chamber and cause standing waves. Standing waves are caused by twowaves that are in phase and traveling in opposite directions. Thecombined effect of the two waves is the creation of antinodes and nodes.The waves perfectly interfere at the nodes to create spots where noenergy is delivered. The waves perfectly cohere at the antinodes tocreate spots where twice the energy of a single wave is delivered. Thewavelength of the radiation is appreciable compared to the speed atwhich heat diffuses within an item that is being heated. As a result,electronic ovens tend to heat food unevenly compared to traditionalmethods.

Electronic ovens are also prone to heat food unevenly because of themechanism by which they introduce heat to a specific volume of the itembeing heated. The electromagnetic waves in a microwave oven causepolarized molecules, such as water, to rotate back and forth, therebydelivering energy to the item in the form of kinetic energy. As such,pure water is heated quite effectively in a microwave, but items that donot include polarized molecules will not be as efficiently heated. Thiscompounds the problem of uneven heating because different portions of asingle item may be heated to high temperatures while other portions arenot. For example, the interior of a jelly doughnut with its high sucrosecontent will get extremely hot while the exterior dough does not.

Traditional methods for dealing with uneven cooking in electronic ovensinclude moving the item that is being heated on a rotating tray andinterrupting the beam of electromagnetic energy with a rotating stirrer.Both of these approaches prevent the application of an antinode of theelectromagnetic waves from being applied to a specific spot on the itemwhich would thereby prevent uneven heating. However, both approaches areessentially random in their treatment of the relative location of anantinode and the item itself. They also do not address the issue ofspecific items being heated unevenly in the microwave. In theseapproaches, the heat applied to the chamber is not adjusted based on thelocation, or specific internal characteristics, of the item beingheated.

SUMMARY OF THE INVENTION

Approaches disclosed herein apply energy to an arbitrary item placed ina chamber for heating using evaluative feedback or deterministicplanning to thereby solve the problem of uneven heating in an electronicoven. In some approaches, the evaluative feedback involves an evaluationof the item by sensing a surface temperature distribution for the itemusing an infrared sensor which is provided to a control system. In someapproaches, the evaluative feedback involves an evaluation of the itemby sensing RF parameters associated with the application of energy tothe item such as an impedance match or return loss. In some approaches,the deterministic planning is conducted using an evaluation of any ofthe parameters discussed herein that are used for evaluative feedback.For example, the deterministic planning can be guided by an evaluationof the surface temperature distribution of the item. The evaluation ofthe surface temperature distribution can be conducted during a discoveryphase, which is conducted ex ante to the actual execution of a plandeveloped by such a deterministic planner, for purposes of obtaininginformation that can be used to generate that plan. The evaluation ofthe surface temperature distribution can also be conducted duringexecution of the plan to determine if the actual heating of the item isnot progressing in accordance with what was expected when the plan wasgenerated.

In some approaches, the actions taken by the control system during theexecution of a plan developed by deterministic planning or during theexecution of an evaluative feedback loop involve applying energy to theitem and altering at least one of the intensity of a source of energyfor the chamber and altering the relative position of the item withrespect to a variable distribution formed by that source of energy. Forexample, a microwave source could form a variable distribution ofelectromagnetic energy in a chamber and the relative position of thatvariable distribution to the item being heated could be adjusted. Thevariable distribution could include nodes and antinodes. The variabledistribution may include local maxima or “hot spots” where energy isapplied to a larger degree than any other adjacent location. The localmaxima will have a relative position with respect to the item. Theaction taken by the control system could include altering the relativeposition from a first position value to a second position value. As usedherein, the term “variable distribution” refers to a variance in a levelof energy across space and not to a distribution of energy that isvarying temporally.

In some approaches, evaluative feedback is used to train a controlsystem to apply energy in an optimal manner to any item placed in thechamber. Evaluative feedback is particularly applicable to the problemof uneven heating for arbitrary items because each action taken by thecontrol system provides training information to the control system. Thisis particularly beneficial in the case of training a control system toheat an item because, unlike in pure data manipulation tasks, eachtraining episode involves an appreciable amount of time. Training acontrol system using a training environment that exists purely in thedigital realm, such as training a control system to play chess againstanother digital system, can involve training episodes that last lessthan a millisecond. However, training a control system using a trainingenvironment that includes physical reactions taking place in the realworld involves constraints set by the actual speed of those physicalreactions. Therefore, evaluative feedback, which provides traininginformation in response to each action taken by the control system, isbeneficial for this particular application because more traininginformation can be obtained in a set period of time.

Some of the approaches disclosed herein also utilize the informationgleaned from the evaluative feedback to train the control system using areinforcement learning training system. Reinforcement learning involvesthe use of an action-value function and an assignment of reward valuesto different states. The action-value function takes the state of thesystem and a potential action to take from that state as inputs andoutputs the potential future rewards that will be obtained from takingthat action. In some approaches, the state could at least partially bedefined by the surface temperature distribution of the item. Forexample, the state could be a matrix of temperature values thatcorrespond to physical locations across either a two dimensional planeor three dimensional volume. The two dimensional plane could be set bythe location of pixels on an infrared camera or it could be the actualsurface area of the item extrapolated from a visual image of the item.Reward values could be calculated using multiple inputs and could bederived from the surface temperature distribution. Rewards can bepositive or negative. For example, higher rewards could be provided forkeeping a variance in the surface temperature distribution low. Negativerewards could be provided for causing an item to char or spill in thechamber.

In some approaches, a plan is generated using a deterministic plannerand an evaluation of certain parameters regarding the operation of theelectronic oven or the item. For example, the electronic oven couldevaluate a cost function to determine a plan for heating the item with aminimum cost. Evaluating the cost function could involve the utilizationof information regarding a surface temperature distribution of the itemin the chamber. The information could be used to extrapolate a plannedsurface temperature distribution for the item. For example, the surfacetemperature distribution of the item in response to a predefinedapplication of energy could be used to extrapolate how the temperaturedistribution of the item would change in response to a plannedapplication of energy to the item that had not yet occurred. Theextrapolated distributions could then be used to evaluate the costfunction associated with that planned application of energy. Theinformation could also be used to determine if an actual surfacetemperature was departing from a planned surface temperaturedistribution in order to monitor the performance of the plan againstexpectations and determine if a new plan should be determined. Theinformation could also be used to generate an estimated plan cost usinga heuristic. For example, the surface temperature distribution of theitem in response to a set of predefined applications of energy under aset of configurations for the electronic oven could be used to providean estimate of how much longer an item would need to be heated to reacha target temperature.

A set of example computer-implemented methods that utilize infraredevaluative control can be described with reference to flow chart 100 andelectronic oven 110 in FIG. 1. Flow chart 100 illustrates a set ofcomputer-implemented methods for heating an item in a chamber such asitem 111 in chamber 112 of electronic oven 110 using infrared evaluativecontrol. The infrared evaluative control can involve evaluative feedbackor obtaining information for a deterministic planner. The methods can beexecuted or administrated by a control system in electronic oven 110.The electronic oven can include a microwave energy source 113 and adiscontinuity 114 in the chamber wall. Microwave energy source 113 canproduce a distribution of energy 115 in the chamber. Discontinuity 114can allow an infrared sensor 116 to sense a surface temperaturedistribution of the item. Sensor 116 could be coupled to discontinuity114 via a waveguide or some other means of coupling the infrared energyto the sensor.

In step 101, the control system of electronic oven 110 takes a firstaction. The action is illustrated as being conducted at time t. Thefirst action alters at least one of a relative position of and anintensity of the distribution of energy 115 in chamber 112 frommicrowave energy source 113. The relative position of distribution ofenergy 115 is defined relative to item 111. Distribution 115 can becaused by the standing wave pattern of microwave energy source 113 asapplied to the chamber. Distribution 115 can also be caused by thetargeted application of energy to the item. An example variabledistribution 115 is illustrated as being applied to item 111. Variabledistribution 115 includes a local maxima aligned with item 111 at point117.

In step 102, a surface temperature distribution for the item is sensedusing an infrared sensor. The infrared sensor could be infrared sensor116 capturing infrared radiation from item 111. Step 102 couldalternatively or additionally involve sensing RF parameters associatedwith the delivery of energy to item 111. In this case, some aspects ofstep 102 could be conducted simultaneously with step 101. However, step102 could also be conducted after the electronic oven completed theexecution of the action in step 101. For example, in the specific caseof the surface temperature distribution being measured during adiscovery phase of the overall heating process, the action in step 101could be the provisioning of energy to the item, and the sensing in step102 could be conducted to measure the response of the item in theimmediate aftermath of the energy delivery.

In step 103, the control system of the electronic oven will evaluate afunction to generate a function output. Information derived from thesurface temperature distribution sensed in step 102 and at least onepotential action for the electronic oven to take can be used to evaluatethe function. The potential action is labeled t+Δt to indicate that itis an action that will be taken in a proximate time step. The loop backfrom step 103 to step 101 is indicative of a time step in which time isincremented by Δt and the next action is executed. During this seconditeration of step 101, the control system of the electronic oven willtake a second action. The second action will alter at least one of therelative position and the intensity of the distribution of energy in thechamber from the microwave energy source. The second action is selectedfrom a set of potential actions based on the function output. As will bedescribed below, the second action can be the next action taken by thecontrol system in accordance with an evaluative feedback loop or it canbe an action taken at a later time as part of a sequence of actions inaccordance with a plan. The plan could be determined by a deterministicplanner or by an optimization analysis conducted in step 103.

The function could be an action-value function F(s,a) with a rewardvalue serving as the function output. The reward value could be a rewardassociated with taking action “a” from state “s.” The informationderived from the surface temperature distribution could be the statevalue “s” for the function. The next action the electronic oven wouldtake could then be the second input “a” to the action-value function.This approach could be used in combination with an evaluative feedbackor reinforcement learning approach as described below.

The function could alternatively by a cost function F(n) with a costvalue serving as the function output. The cost value could be a plancost associated with executing a plan to heat the item based on anevaluation conducted at node “n.” The node can be defined by a sequenceof actions that the electronic oven is capable of executing. One of theactions in that sequence of actions is the potential action utilized toevaluate the function. The plan cost can be associated with a traversedplan cost resulting from the execution of that sequence of actions. Thenode could also be associated with an extrapolated state of the itemprovided by an extrapolation engine. The node could also be associatedwith an estimated future plan cost provided by a heuristic. Theinformation derived from the surface temperature distribution could beused by the extrapolation engine to extrapolate the extrapolated state.The information derived from the surface temperature distribution couldalso be used to determine if a deviation has occurred from theextrapolated effect of the plan. For example, an extrapolated surfacetemperature distribution that was expected to result from a sequence ofactions could be compared against an actual surface temperaturedistribution that was sensed after the actual execution of thoseactions. The evaluation could be conducted by a deviation detector. Upondetecting a deviation, the control system could abandon the originalplan and generate a new plan. These approaches could be used incombination with a deterministic planner approach as described below.

The function could alternatively be executed by an optimization analysissolver. The optimization analysis could determine if it was possible toproduce a plan to heat the item to a target state within an acceptableerror value (i.e., tolerance). The analysis could be conducted using thedata obtained from a sensor during the execution of a previous action.For example, the data could be collected in sensing step 102 and definea response of the item to an action conducted in step 101. Theoptimization analysis could then determine if previously conductedactions, for which a response was known through the obtained sensordata, could be repeated in a particular sequence to bring the item froma current state to a target state. The optimization analysis could use aconvex optimization solver. The output of the optimization analysiscould be used to directly derive a plan to heat the item. In that case,the actions conducted in future iterations of step 101 could involve theexecution of actions specified by the plan. Such an optimizationanalysis could, also or alternatively, be used as the extrapolationengine or heuristic described above and thereby as part the plangeneration process of a deterministic planner.

Some of the approaches disclosed herein involve an alteration to acontrol or training system based on the identity of the item placed inthe chamber or a particular kind of heating selected by a user. Inparticular, the action-value function, cost function, heuristic,extrapolation engine, deviation detector, state characteristics, rewardderivation procedure, optimization analysis, or training system for areinforcement learning approach mentioned herein could be altered basedon the identity of the item or commands from the user. For example, thecost function and reward derivation procedure could both be altered ifan item was known to recovery slowly from major temperature disparitiessuch that keeping an even temperature was associated with greaterrewards and lower costs. As another example, the tolerance for theoptimization analysis could be decreased if the item was recognized asone that dried out rapidly or charred if a target temperature wasexceeded for a short period of time.

In these approaches, information concerning the identity of the item canbe provided by various channels to assist the operation of the controland training systems. The channels can include a QR code or UPC barcodelocated on a package of the item. Another channel could be a response tothe item to a given calibration step such as a monitored response of theitem to an application of energy as monitored by an infrared sensor.Another channel could be a separate machine learning algorithm such as atraditional neural network trained to work as a classification system toidentify items in the chamber as specific food items. Another channelcould be the visible light reflected off the item and detected by avisible light sensor. Another channel could be an input from a user ofthe electronic oven via a user interface. The information provided viathese channels could simply identify the item and allow the controlssystem to determine how to alter itself, or the information couldactually be directly applied to change the control system. For example,a QR code could identify an item as a frozen dinner, and the controlsystem could load a new reward derivation procedure based on theidentification information, or the information in the QR could itself bea new reward derivation procedure. For example, the reward derivationprocedure could reward a gradual phase change in the item from frozen tomelted.

Some of the approaches disclosed herein require the definition of atarget state in order for an associated planner or reinforcementlearning system to function. The target state can be received from auser of the electronic oven at varying degrees of specificity. Forexample, the user could specify a specific temperature for the item or aset of different temperatures for sub-items. Alternatively, the usercould specify a general command such as “warm” or “boil” and the targetstate could be derived from that command. The target state could bedefined by a temperature distribution across the surface of the item orthroughout the volume of the item. In some approaches, the electronicoven will intuit the target state from context such as an identity ofthe item, prior inputs received from the user, and other externalfactors such as the location of the electronic oven and the time of day.

The disclosed approaches improve the fields of electronic ovens andmicrowave heating by providing more reliable heating. Controlling theapplication of electromagnetic energy to an item in a controlled andreliable manner is a technical problem. The disclosed approaches includea set of aspects that contribute to a solution to that technicalproblem. In particular, the use of evaluative feedback, optimizationanalyses, deterministic planners, and reinforcement learning asdescribed herein each enhance the accuracy and efficiency of a controlsystem for heating an item in an electronic oven in an inventive mannerto solve the aforementioned technical problem and improve the operationof electronic ovens generally.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 includes a flow chart for a set of computer-implemented methodsfor heating an item in a chamber using an evaluative control system andan illustration of an electronic oven in accordance with approachesdisclosed herein.

FIG. 2 includes a plan view and side view of a reflective element foraltering a distribution of energy in the chamber of an electronic ovenin accordance with approaches disclosed herein.

FIG. 3 includes a flow chart for a set of computer-implemented methodsfor heating an item in a chamber using an optimization analysis inaccordance with approaches disclosed herein.

FIG. 4 includes a set of color coded grids that reflect state vectorsand response vectors which facilitate a description of some of theoptimization analyses in FIG. 3.

FIG. 5 includes a data flow diagram that facilitates a description ofsome of the optimization analyses in FIG. 3.

FIG. 6 includes two sets of axes that chart two plans derived from asingle duration vector in accordance with approaches disclosed hereinwhere the x-axis of both sets of axes is time in units of seconds andthe y-axis of both sets of axes is temperature in units of degreesCelsius.

FIG. 7 includes a set of axes that chart a simulated error of anoptimization analysis where the x-axis is a number of configurationsavailable to an optimization analysis and the y-axis is the error indegrees Celsius.

FIG. 8 includes a flow chart for a set of computer-implemented methodsfor heating an item in a chamber using an evaluative feedback controlsystem with reinforcement learning in accordance with approachesdisclosed herein.

FIG. 9 includes a block diagram illustrating the operation of a controlsystem with a function approximator serving as an action-value functionin accordance with approaches disclosed herein.

FIG. 10 includes a flow chart for a set of computer-implemented methodsfor heating an item in a chamber using a deterministic planner inaccordance with approaches disclosed herein.

FIG. 11 includes a conceptual diagram of an extrapolation engineutilizing a surface temperature distribution to extrapolate a state ofthe item in response to a set of actions in accordance with approachesdisclosed herein.

FIG. 12 includes a conceptual diagram of the performance of a derivedplan is monitored as it is executed in accordance with approachesdisclosed herein.

FIG. 13 includes a data flow diagram illustrating a control system foran electronic oven in accordance with the reinforcement learningapproaches disclosed herein.

FIG. 14 includes a data flow diagram illustrating a control system foran electronic oven in accordance with the deterministic plannerapproaches disclosed herein.

FIG. 15 includes a conceptual diagram of an aspect of a state derivationsystem in accordance with approaches disclosed herein.

FIG. 16 includes a data flow diagram illustrating the initialization ofthe control system in FIG. 13 using data from external channels.

DETAILED DESCRIPTION OF THE EMBODIMENTS

Control systems that use evaluative control to heat an item in a chamberof an electronic oven are disclosed. In some approaches the controlsystems use evaluative feedback. The output of the control system caninclude a power level and a relative position of a variable distributionof electromagnetic energy applied to a chamber of the oven with respectto an item in the chamber. The feedback to the control system cancomprise visual light data, a surface temperature distribution of theitem, or RF parameters associated with the absorption of electromagneticenergy by the chamber or item. In some approaches, the evaluativefeedback is used to train the control system using a reinforcementlearning training system. In some approaches, the control systemgenerates a plan using a deterministic planner to heat the item. In someapproaches, the evaluative feedback is used to learn the response of theitem to a given action. The pairs of responses and actions can then beused to derive a plan to heat the item to a target state. The plan caninclude a sequence of actions that change the power level and variabledistribution for the pattern of electromagnetic energy applied to thechamber relative to an item in the chamber. The plan can be generatedbased on an evaluation of a surface temperature distribution of the itemduring a discovery phase of the overall heating process and anextrapolation of how the surface temperature distribution would alter inresponse to additional actions taken by the electronic oven. Theperformance of a plan can be monitored based on an evaluation of asurface temperature distribution of the item, or other feedbackparameter, during execution of the plan.

Electronic Oven Components

Electronic oven 110 in FIG. 1 illustrates various features of anelectronic oven that can be used in accordance with approaches disclosedherein. The oven opening is not illustrated in order to reveal chamber112 in which item 111 is placed to be heated. Item 111 is bombarded byelectromagnetic waves via a variable distribution of energy 115 from anenergy source 113. The item can be placed on a tray 118. Electronic oven110 includes a control panel 119. The control panel 119 is connected toa control system located within oven 110 but outside chamber 112. Thecontrol system can include a processor, ASIC, or other embedded systemcore, and can be located on a printed circuit board or other substrate.The control system can also have access to firmware or a nonvolatilememory such as flash or ROM to store instructions for executing themethods described herein.

Energy source 113 can be a source of electromagnetic energy. The sourcecould include a single wave guide or antenna. The source could includean array of antennas. The electromagnetic waves can be microwaves. Theelectronic oven 110 can include a cavity magnetron that producesmicrowaves from direct current power. The microwaves could have afrequency of 2.45 GHz or 915 MHz. The cavity magnetron can be powered bymodern inverter microwave technology such that microwaves can beproduced at varying power levels. However, traditional powerconditioning technology can be used to produce a set level of directcurrent power for the magnetron. The electromagnetic waves could beradio frequency waves generally. The frequency of the waves could alsobe alterable by energy source 113. Energy source 113 could also beconfigured to produce multiple wave patterns with different frequenciessimultaneously.

Microwave 110 can form a variable distribution 115 in chamber 112 withantinodes and nodes formed at different three dimensional points withinthe volume defined by chamber 112. A particular physical configurationfor the variable distribution within chamber 112 can be referred to as amode of the electronic oven. The relative distribution of energy in thechamber can be altered by altering the mode of the electronic oven whilekeeping item 111 stationary, or by moving item 111 within the chamber.The energy source 113 can include a mode stirrer to prevent theformation of standing waves in fixed positions within chamber 112. Themode stirrer can be a collection of protrusions placed in such a way asto partially obstruct the electromagnetic energy being applied tochamber 112, and to alter the manner in which the energy is obstructedto cause varying degrees of reflection and alter the pattern ofantinodes and nodes formed in chamber 112. In situations where theenergy source 113 is an array of antennas, the energy source couldinstantaneously deliver variable levels of energy to the antennas in thearray to alter the variable distribution within chamber 112.

Control panel 119 can be used to communicate with the user. The controlpanel 119 is used to provide information to the user, receive commandsfrom the user, or both. Control panel 119 is shown with an optionaldisplay, keypad, speaker, and camera. The control panel could displayinformation on the display. The display could be touch enabled andreceive commands from the user via a touch controller. The control panelcould provide audio prompts via the speaker and receive voice commandsfrom the user via an integrated microphone. Depending on thesophistication of the voice system, the speaker can also be used tocarry out a basic dialog with the user to guide them in the entry ofvoice commands to the electronic oven. The control panel could receivecommands from the user via the keypad. Although a basic set of keys arepresented in FIG. 1, the electronic oven could have any number ofspecialized keys for inputting commands specific to certainfunctionalities of electronic ovens disclosed herein. The control panelcould receive gesture commands from the user via the camera or throughan alternate ultrasound or ultraviolet sensor. The control panel couldreceive information from UPC or bar codes from the packaging of itemsbefore they are placed in chamber 112 via the camera. The camera canalso be configured to recognize items placed into the field of view ofthe camera using traditional classifier and image recognitiontechniques.

Electronic oven 110 could also include one or more connections to awired or wireless communication system. For example, the oven couldinclude a radio for a satellite or Wi-Fi connection. The control systemfor electronic oven 110 could include a web browser or simple HTTPclient for communicating over the Internet via that radio. The wirelesscommunication system and control system could also be configured tocommunicate over a LAN or PAN such as through the use of Bluetooth,Zigbee, Z-wave or a similar standard. The radio could also be configuredto conduct inductive communication with RFID tags placed on thepackaging of items to be heated. The inductive communication could beNFC communication.

The electronic oven could communicate via any of the aforementionedmeans to a central server administrated by or on behalf of themanufacturer of electronic oven 110 to receive updates and provideinformation on the machine's operation. All of the functionalityprovided by control panel 119 can be provided by a separate consumerdevice such as a mobile telephone or web portal on a workstation via anyof the aforementioned means. Communication could include providingstatus information from the oven to the device or commands from thedevice to the oven. Additional functionality may be provided given thepotential for the device and oven to be in separate places (e.g., morefrequent status updates or a visible light image of what is in thechamber).

Electronic oven 110 can also include a discontinuity in the walls ofchamber 112 that is configured to allow electromagnetic radiation tochannel out of the chamber. The discontinuity could be an opening 114.Although opening 114 in the electronic oven is shown on a wall ofchamber 112, the opening can be located anywhere on the surface ofchamber 112 which provides a sufficient view of the interior of chamber112. The opening could comprise a past cutoff waveguide with physicalparameters set to block the electromagnetic energy from energy source113 while allowing electromagnetic energy in other spectrums to escapethrough opening 114. For example, microwave energy could be preventedfrom exiting the opening while visible light and infrared energy wereallowed to pass through opening 114.

Opening 114 could channel the energy from chamber 112 either directly orthrough a waveguide, to a sensor. The sensor could be configured todetect infrared energy or visible light, or a combination of the two.The sensor or set of sensors could include an IR camera, a visible lightcamera, a thermopile, or any other sensor capable of obtaining visiblelight sensor data and/or infrared light sensor data. In a specificexample, the opening could be connected to a standard visible lightcamera with an IR filter removed in order for the camera to act as botha visible light sensor and an infrared sensor and receive both infraredsensor data and visible light sensor data. A single sensor approachwould provide certain benefits in that an error in the alignment of twodifferent fields of view would not need to be cancelled out as could bethe case with a two sensor system.

The same opening could be used to channel both visible and infraredlight out of the chamber. In one approach, a time multiplexed filteringsystem, which could be used additionally or in the alternative to thepast cutoff waveguide, could allow a single sensor, or multiple sensors,to detect both visible light and infrared energy from the same stream ofelectromagnetic energy. The filter could comprise a wheel, or otherselector, with filters for different spectra of electromagnetic energy.The wheel would be placed in line with the stream of electromagneticenergy and alternatively transmit solely the visible light or infraredenergy. A sensor, or sensors, placed on the alternative side of thewheel would then be able to detect the desired light from the incomingstream. The sensor could also be configured to obtain informationregarding both spectra and resolve the signal into its infrared andvisible light component part using digital filtering. In anotherapproach, the sensors could be configured to continuously obtaindifferent segments of the same stream of electromagnetic energy by, forexample, being positioned at slightly different angles with respect toan opening on the chamber.

An example electronic oven can also include additional openings in orderto obtain different views of item 111. Data from the various views couldthen be combined to form a three-dimensional image of the item. However,a camera applied to sense visible light through opening 114 couldalternatively be a three-dimensional camera to achieve a similar result.In particular, two openings can be utilized with two cameras to obtainstereoscopic information regarding item 111. As another particularexample, the two openings could be used to obtain different streams ofdata (e.g., opening 114 could obtain a stream of visible light sensordata while another opening obtained a stream of infrared light sensordata).

An example electronic oven in accordance with this disclosure couldinclude other features not illustrated in FIG. 1. The oven could beaugmented with numerous additional sensors. The sensors could includetemperature sensors, auditory sensors, RF parameter sensors, humiditysensors, particulate concentration sensors, altitude sensors, ultrasoundsensors, ultraviolet or IR sensors, a weight sensor such as a scale, andany other sensors that can be used to obtain information regarding thestate of the item, chamber, or oven. For example, the oven could includesensors to detect the power applied to chamber 112 via source 113, thereturn loss from chamber 112, an impedance match between the energysource and item or chamber, and other physical aspects of the energysource. In particular, the return loss can be measured to determine aphase change in item 111 as certain items absorb energy at a muchgreater degree when they are melted compared to when they are frozen.Impedance matching or return loss measurements could also be applied todetect more subtle changes in the physical characteristics of the itembeing heated. Additional sensors could detect the humidity of the airexiting chamber 112 via a ventilation system or within the chamber.Additional sensors could detect a particulate concentration within thosevolumes to determine if the items were smoking. Additional sensors coulddetect the weight of item 111.

Electronic oven 110 can include a transparent cover to place over item111 to prevent splattering within chamber 112 as item 111 is heated. Thecover could be transparent to both visible light and to infrared lightso as to not interfere with the sensing of electromagnetic radiation inthose frequency bands via opening 114. For example, the cover could beinfrared transmitting plexiglass. The cover could also be treated toprevent the formation of condensation by coating the material in ahydrophobic layer or by creating perforations in the cover to allowmoisture to escape the enclosure.

A specific class of approaches for modifying the variable distributionof energy in the chamber involves applying energy from an energy sourceto a set of variable reflectance elements. The reflectance of theelements can be altered to introduce a different phase shift to incidentelectromagnetic waves from the energy source. Examples of suchapproaches are described in U.S. Patent Application Nos. 62/434,179,filed Dec. 14, 2016, and entitled “Electronic Oven with ReflectiveEnergy Steering,” and 62/349,367, filed Jun. 13, 2016, and entitled“Electronic Oven with Reflective Beam Steering Array,” both of which areincorporated by reference herein in their entirety for all purposes. Thestates of the variable reflectance elements, and the state of the energysource, can define different configurations for the electronic oven. Theconfigurations can each be associated with a different mode or variabledistribution of energy in the chamber. As such, the differentconfigurations will result in a different distribution of energy beingapplied to an item in the chamber. Selecting from among the differentconfigurations will therefore result in different heating patterns forthe item and can allow the oven to heat different portions of the itemdifferently or to more uniformly heat the item as desired.

The different configurations are defined by different associatedvariable distributions of energy being produced within the chamber.However, the configurations do not necessarily require the electronicoven itself to take on different physical configurations. In someapproaches, the state of the variable reflectance elements and theenergy source can be altered without the electronic oven utilizing anymoving parts. For example, the variable reflectance elements and energysource could each solely comprise solid state devices, and theconfiguration of the oven could be set by providing different signals tothose solid state devices. However, in other approaches, theconfigurations will involve different physical configurations for theelectronic oven. For example, the variable distribution of energy in thechamber can be altered by independently altering the physical positionof variable reflectance elements in a set of variable reflectanceelements as described with reference to FIG. 2.

The electromagnetic waves applied to the chamber, such as from microwavesource 113, can be a polarized or partially polarized electromagneticwave. Therefore, by altering the orientation of a variable reflectanceelement upon which the electromagnetic wave is incident, thedistribution of energy in the chamber can be altered. In particular, theposition of the reflective elements can be altered to adjust theorientation of the reflective element with respect to the dominantpolarization of an electromagnetic wave in the chamber. For example, thephase shift introduced by each variable reflectance element could bealternately changed in a binary fashion from 0° to 90° and back, orcould be altered in an analog fashion anywhere from 0° to 180° with asmooth transition between each gradation on the spectrum. As a morespecific example, the orientation of each variable reflectance elementwith respect to the dominant polarization of an incident electromagneticwave could be changed from 0° to 90° and back, or could be anywhere from0° to 180° with a smooth transition between each orientation. Notably,even in the binary case, the variable reflectance element may be just asingle element in a large set, such that a large degree of flexibilitycan still be provided to the control system despite the fact that eachindividual element only has two states.

FIG. 2 illustrates a variable reflectance element 200 both from a sideview (top image of FIG. 2) and plan view (bottom image of FIG. 2).Element 200 alters a variable distribution of energy in the chamber byaltering its physical position from a first position to a secondposition. Element 200 includes a reflective element 201 which in thiscase is a relatively flat piece of conductive material that could beformed of sheet metal such as aluminum, steel, or copper. The reflectiveelement 201 is held above a surface of the chamber, defined by chamberwall 202, by a dielectric axle 203 that extends through a discontinuity204 in the chamber wall. The axle is dielectric, passes through a smallperforation, and is generally configured to avoid creating an antennafor microwave energy to leak out of the chamber.

A motor on the exterior of the chamber is able to rotate reflectiveelement 201 via dielectric axle 203 by imparting a force to the axle asillustrated by arrow 205. The force could be applied by a rotor attachedto axle 203. The entire structure illustrated in FIG. 2 could be sealedbehind a false wall of the chamber to shield the structure from stainsor mechanical damage. The motor is able to rotate the axle between a setof positions selected from a fixed set of positions. For example, themotor could adjust the axle so that the reflective element 201 wasrotated back and forth through a 90° arc. However, the motor could alsorotate the reflective element through any number of fixed steps along anentire 360° degree arc.

Many of the approaches described above exhibit the feature of theelectronic oven being capable of being placed in multiple configurationswhile the control system keeps track of which configuration theelectronic oven is in with precision. In contrast to the operationparadigm of traditional mode stirrers and similar devices in which thecontrol system is unaware of the current state of the electronic oven,the disclosed approaches allow the control system to independently alterthe state of multiple elements and set the state of the oven withparticularity. This allows the control system to effectively executemany of the control approaches described below because the electronicoven has numerous states available to it, and can observe withspecificity the response of the item to a particular application ofenergy in those particular states.

Evaluative Feedback—Heuristic Optimization Control

The response of an item in the chamber to a given action can be sensed,evaluated, and stored as a description of how the item responded to thataction. These steps can be repeated to form a library of descriptions ofhow the item responds to various actions. The library of descriptionscan then be utilized to develop a plan for heating the item from acurrent state to a target state. The plan can be developed automaticallyby the control system using an optimization analysis that selectsactions from the library to drive the item from a current state towardsa target state subject to various constraints. The control system canalso determine if there is no plan capable of reaching the target statewithin a given set of constraints with the level of informationcurrently known. At that point, the optimization system can indicate theneed to obtain another description of how the item responds to anadditional action.

The approach outlined in the previous paragraph will not always providean accurate description of how the state of the item will change throughthe application of various actions. This is because the system isgenerally not time invariant. The execution of an action upon an itemduring a heating task tends to change the item, sometimes dramatically,and tends to alter the response the item will experience when such anaction is repeated. In a basic example, an observation of how a cube ofice response to a blast of heat will be different than an observation ofhow that item responds to the same blast of heat when it has alreadymelted because the different phase of the item cause the item to exhibitdifferent heat specificity. As a result, plans generated using thedescriptions in the library will be more accurate in the near term asthe characteristics of the item have not changed to an appreciabledegree from when the response was sensed and recorded.

Given the time variant nature of the analyses described in this section,the analyses can be considered methods for developing a heuristicdescription of how the item will respond to a heating plan. However, theanalyses can be performed with relatively little time and resources ascompared to other machine intelligence techniques. Therefore, theanalyses can be run with relatively high frequency compared to theactual execution of actions by the electronic oven, and variations fromthe state expected by the original estimate can be continuouslycorrected for. In effect, the control system: (i) can operate usingplans produced by analyses, which are likely to be at least somewhataccurate in the near term; and (ii) can be continuously producingupdated plans through additional iterations of the analyses, whichreplaced prior plans as the accuracy of those prior plans begin todecline. These additional iterations will assure that the expected andactual performance of each plan does not diverge to an unacceptabledegree. The control system can also be updated to reflect differences inhow the item responds to a given action by discarding previously storedresponses in the library so that the optimization analyses operates onupdated information regarding the response of the item.

The frequency at which the additional iterations of the analysis isconducted should be controlled to assure that the accuracy of the planthat is currently being executed is maintained while at the same timeallowing the effect of a given plan to be sensed and registered by thecontrol system. Given a system in which the execution of each action ison the order of seconds, including the overhead associated withtransitioning between configurations, and given that the accuracy of theheuristic's prediction tends to fall off on the order of tens ofseconds, the period between additional iterations of the analyses shouldbe greater than 3 seconds and less than 15 seconds.

A set of computer-implemented methods for heating an item in a chamberof an electronic oven towards a target state can be described withreference to flow chart 300 in FIG. 3. Flow chart 300 includes step 301of heating the item with a set of applications of energy to the chamberwhile the electronic oven is in a respective set of configurations. Theapplications of energy and the respective set of configurations define arespective set of variable distributions of energy in the chamberrelative to the item in the chamber. The variable distributions ofenergy are referred to as being variable because the level of energythrough the physical space of the chamber is variable, not because thedistributions vary temporally. A given variable distribution of energyrelative to the item must be maintained for enough time for the systemto detect a response that is directly attributable to that variabledistribution. The configurations can involve varying the manner in whichenergy is directed from the energy source to the item, a relativeposition of the item with respect to the chamber, and a physicalconfiguration of the electronic oven itself. In keeping with FIG. 2, thedifferent configurations could be distinguished by rotating one or morevariable reflectance elements in a set of variable reflectance elements(i.e., variable reflectance element 200 could be rotated 90° totransition from one configuration to another).

Step 302 involves sensing sensor data that defines a respective set ofresponses by the item to the set of applications of energy. The set ofapplications of energy are distinct variable distributions of energy inthe chamber relative to the item as caused by different configurationsof the electronic oven and applications of energy to the chamber. Thesensor can be an infrared sensor, or any of the sensors describedherein. The responses are respective in that each response defines theresponse of the item to a specific, respective, application of energyand configuration of the electronic oven. The respective responses,applications of energy, and configurations are combined to make a set ofentries in the library of responses described above. The configurationscan be different physical configurations of the electronic oven. Forexample, a first response could be “temperature increased 2 degrees F.”and correspond to a respective application of energy “50%” and arespective physical configuration “nominal,” while a second responsecould be “temperature increased 5 degrees F.” and correspond to arespective application of energy “100%” and a respective physicalconfiguration “tray rotated 30 degrees.” The configurations can begreatly varied based on the complexity of the electronic oven. Forexample, a large vector of different rotation values could be requiredto describe the configuration of an electronic oven having a large arrayof elements similar to reflective element 200 in FIG. 2.

The duration of each application of energy and the commensurate time inwhich the electronic oven is in a given configuration must be controlledto assure that the control system is able to accurately attribute agiven set of sensor data with a particular variable distribution ofenergy in the chamber relative to the item. If the configurations changetoo rapidly, there is no way to assure that the recorded sensor data isan accurate representation of how the item responded to thatconfiguration. Specifically, the sensor data obtained in step 302 shouldcorrespond to a known application of energy and a known configuration sothat a repetition of the associated variable distribution of energy inthe chamber relative to the item at a later time will produce a knownresult when the plan is executed. However, the sensor data may or maynot be collected simultaneously with the application of energy. Indeed,in certain applications, the sensor data will be collected immediatelyafter the application of energy. Regardless, the duration of applicationshould be selected to allow the library to store the sensor data inassociation with data representing the variable distribution of energyin the chamber relative to the item.

The duration for each application of energy and associated configurationdepends in part on how fast the electronic oven can shift betweenconfigurations. This factor is important for implementations thatutilize different physical configurations. In certain approaches, thisrequirement is met by assuring that the chamber remains whollymotionless during each application of energy. For example, if thechamber includes a set of variable reflectance elements that arephysically adjusted to alter the variable distribution of energy in thechamber relative to the item, the elements remain motionless during theapplication of energy. Here, the chamber is defined as the region ofmaterial in which electromagnetic energy reflects to define the mode ofthe electronic oven (e.g., false walls that are transparent toelectromagnetic energy do not define the chamber area that must remainmotionless). Furthermore, in situations in which the configurations arephysical configurations, and energy is being continuously supplied tothe chamber, each corresponding application of energy should be at least0.5 seconds in duration to allow the electronic oven to transition froma prior configuration, and exhibit an independently measurable responseon the item. This estimate assumes an ability of the electronic oven totransfer between configurations in 0.1 seconds or less, and the durationof each application of energy should be increased in lock step with thetime it takes to shifting between configurations if doing so takeslonger than 0.1 seconds. The different applications of energy could bepart of a continuous application of a uniform amount of energy so longas each individual application of energy was independently attributableto a respective configuration of the electronic oven and correspondingresponse.

Steps 301 and 302 can be conducted during a discovery phase that isintended to discover information before the actual execution of a plan.However, steps 301 and 302 can also be conducted as part of theexecution of a previously generated plan. In addition, steps 301 and 302can involve an additional discovery phase conducted after a plan hasbeen partially executed. The benefit of a discovery process is that itmay be easier to analyze the response of the item to a specificapplication of energy and configuration since it can be analyzed inisolation as opposed to being conducted in sequence with other steps ina plan. For example, the electronic oven could be placed in a givenconfiguration and allowed to settle before applying an application ofenergy. As a result, the response sensed in step 302 will be an accuratedescription of how the item responded to that particular application ofenergy without second order effects caused by subsequent or proximateapplications of energy.

An additional benefit of conducting the sensing during a discoveryphase, is that it will obtain information that may already be needed forthe electronic oven to provide certain functionality. For example, ifthe sensing for step 302 is conducted during a discovery phase, the samesensor data could be used to identify the item and could optionally beused to segment the item. In particular, the identity of the item couldbe determined using a classifier operating on data reflecting theresponse of the item to a given application of heat. As different itemsrespond to an application of heat differently and different classes ofitems responds similarly, a classifier can be trained on this data toidentify an item. Therefore, the same process to identify an item can beused to collect data to develop a plan to heat the item to a targetstate.

Segment step 303 comprises segmenting the item into a set of segments.The segments can be used to guide the generation of the plan in step 304and measure the responses obtained in step 302. To this end, theelectronic oven can be augmented with image processing systems thatallow the control system to keep track of the actual physical locationof the segments regardless of whether the item is moved within thechamber. This functionality can be aided by any of the means of sensingvisible light disclosed herein. The number of segments can be tailoredin light of the fact that a large number of segments will increase thecomputational complexity and resource consumption required to carry outsteps 302 and 304, while a small number of segments might not providesufficient information for assuring that the item is heated evenly asrequired. The segments can each be defined by a center point and anarea. The center point can be referred to as a point of interest on theitem.

The number of segments, location of center points, and areas of thesegments can all be set properties of the control system, or could beadjusted based on the characteristics of the item in the chamber. Forexample, items with large heat resistivity could be identified and theidentity of the item could be used to set the number of points ofinterest high and the areas of the segments low. This approach likewisebenefits from the fact that the same data can be used to segment theitem as to identify the item because there is no overhead in terms ofphysical actions that must be conducted by the electronic oven to obtainthis information.

The different segments can be drawn to points of interest on the item innumerous ways. The segments can be set by a physical location in theelectronic oven where different locations for the segments areidentified according to a uniform pattern across the chamber. Forexample, segment 1 could be a square inch of the bottom of the chamberin the back left corner etc. However, the location of the segments canalso be guided to only track portions of the item added to the chamber.For example, portions of the chamber that don't respond to theapplication of energy during a discovery phase can be ignored whileregions that did respond could be identified as portions of the item inthe chamber and selected as segments.

The segments can be set to cover a set area around a given point ofinterest, or they can encapsulate areas with varying size. For example,the segments could be configured to cover areas of the item that exhibita similar response to heat. Sensor data collected in step 302 couldindicate that the item actually included three different sub-items thatrespond to temperature in three distinct ways. In this example, theelectronic oven could be discovering that the item has three sub-itemscorresponding to a meal of protein, vegetable, and starch. The sub-itemscould then be used as a basis for segmentation in which each sub-itemwas treated as a segment or collection of segments.

Step 303 is drawn with phantom lines in a feedback loop from step 302because the segmenting step does not necessarily have to be conductedusing an evaluation of sensor data conducted during execution of theplan. Instead, the segmentation step can be conducted prior to step 301using a classifier and visual light data, user input, or any otherchannel for information from an external source to the electronic ovendescribed elsewhere. For example, a user could directly provide input toa control system of the electronic oven to manually segment the item.

In step 304, a plan is generated to heat the item in the chamber. Thegenerating is conducted by the control system of the electronic oven andutilizes the sensor data obtained in step 302. The sensor data can beused in the sense that responses from the library, as obtained from thesensor data, are analyzed and pieced together to create a plan for goingfrom a current state to a target state. The pieced together responsesthat lead from the current state to the target state end up forming theplan because the responses are stored in the library along with theapplications of energy and configurations of the electronic oven thatproduced those responses. Therefore, the plan will be a sequence ofcommands that place the oven in those configurations and applies thoseapplications of energy. The responses from the library can be selectedusing an optimization analysis. Multiple copies of each response can beselected to make a single plan.

The responses selected from the library can be sequenced using theoptimization analysis, or can be sequenced using a separate process. Assuch, the generation of the plan in step 304 can involve two steps of:conducting an optimization analysis to produce an optimization output305, and compiling a sequence of commands using the optimization output306. The optimization output can include an error value and a vector.The error value can be a scalar temperature value in degrees Celsiusindicating an expected deviation in temperature between an expectedfinal state and the target state. The vector can describe the responses,and the associated variable distributions of energy in the chamberrelative to the item, that will be utilized to generate a plan forheating the item. The sequence of commands compiled in step 306 candefine an order for applying different variable distributions of energyrelative to the item to the chamber. To this end, the commands cancontrol the applications of energy to the item and alter theconfiguration of the electronic oven. The electronic oven can thenexecute the sequence of commands to heat the item towards the targetstate.

An evaluation of an error value generated in step 305 can be utilizeddetermine if additional entries for the library need to be obtained, orif the system should proceed with the execution of a plan. For example,if the error value exceeds an acceptable error value, the process canreturn to step 301 to obtain more response data and execute anadditional iteration of step 304. In addition, the process can skip step306 upon detecting that the error value exceeds an acceptable errorvalue. The illustrated process in FIG. 3 is also general enough toinclude a loop back to step 301 even if the error value does not exceedan acceptable error value, as additional iterations of steps 301 and/or302 can be conducted while the electronic oven is executing a plan thatis associated with an acceptable error value (i.e., a plan that isexpected to perform within an acceptable tolerance).

Step 305 can involve the use of a solver, data representing the targetstate, and data representing the set of responses obtained in step 302.Data representing the target state can be obtained from a user, beautomatically generated by the control system, or be received via anexternal channel. The data representing the set of responses can betaken from the library, as stored in combination with data representingthe variable distribution of energy in the chamber relative to the itemthat caused those responses. The data representing the set of responsesand data representing the target state can be a set of temperaturevalues or a set of temperature derivatives. The data can includemultiple data points to represent multiple segments of the item. Thedata could correspond to the surface temperature of the item. The solvercan be a convex optimization solver. The convex optimization solver cansolve for the set of responses that would take the item from a currentstate towards the target state. The solver can be subject to numerousconstraints such as minimizing overall heat time, minimizing temperaturevariation across the item or across groups of segments of the item, orminimizing a maximum temperature of any segment on the item. The solvercan solve for the vector and that generates a minimum error valuesubject to the constraints of the optimization analysis.

A specific class of implementations of step 305 can be described withreference to FIGS. 4 and 5. In these approaches, an optimizationanalysis is utilized to produce a duration vector, and may also producean error value. The duration vector can include a set of elements thatrepresent a duration for a respective set of applications of heat andconfigurations of the electronic oven that will bring the item from acurrent state to a target state. The error value quantifies a differencebetween the target state and an extrapolated end state. The extrapolatedend state could be calculated using the duration vector and a set ofresponse vectors as described below. The optimization analysis couldutilize, a solver, such as a convex optimization solver, to select theduration vector so as to minimize the error value. The duration vectormay include information regarding the sequence of how variousapplications of energy and configurations of the electronic oven shouldbe applied. However, the sequence could also be selected in a separatestep using the duration vector.

FIG. 4 includes multiple grids with eight cells each. The cells areillustrations of the segments of an item that has been placed in anelectronic oven. The regular nature of the segments is provided forexplanatory purposes, and in an actual application the segments may beof varying sizes and may have irregular shapes. The segments may also bethree dimensional volumes as opposed to two dimensional surfaces. FIG. 5is a data flow diagram to illustrate various optimization analyses thatcan be conducted in combination with the states and responses of theitem in FIG. 4.

Grid 400 provides an illustration of a target state for the item. Asillustrated, the goal of this specific heating task is to heat all eightgrids, but to heat the left four grid squares to a higher temperaturethan the four grid squares on the right. Grids 401, 402, and 403 provideillustrations of the response of the item to respective applications ofheat and configurations of the electronic oven: response 1, response 2,and response 3. Grids 404 and 405 are illustrations of extrapolated endstates for the item that are expected to be reached after the executionof different plans generated using the aforementioned applications ofheat and configurations of the electronic oven. Extrapolated state 404is an extrapolated state that is expected to result from the executionof plan 1. Plan 1 comprises an application of the conditions that leadto response 1 and a following application of the conditions that lead toresponse 2. Grid 406 is an illustration of an error between extrapolatedstate 404 and target state 400. The segments in each state of FIG. 4 areshaded to represent an average surface temperature of the segment, oraverage change in surface temperature of the segment, in which darkshading represents a high temperature/large temperature variation, andlight shading represents a low temperature/slight temperature variation

In general, data representing the target states, response vectors,extrapolated states, and error of a given plan can be numerical valuesorganized into a vector with each element of the vector corresponding toa segment of the item. A target state of the item could include a targetvector with numerical values to represent a target condition for each ofthe segments of the item. For example, data representing the targetstate 400 could include a target state vector with eight numericalvalues to represent the average surface temperature of each segment.Data representing the response of the item to a respective applicationof heat and electronic oven configuration could take a similar format.The response of the item to an application of heat could be a responsevector with a numerical value to represent a change in the averagesurface temperature of each segment in response to a given applicationof heat while the electronic oven was in a given configuration. Ingeneral, the values of the response vectors could comprise anytemperature derivative indicating the response of the item to a selectedapplication of energy. For example, data representing responses 401,402, and 403 could include three response vectors each corresponding toa respective configuration of the electronic oven and application ofenergy, and each with eight numerical values for a temperaturederivative for the segment (e.g., 10 degrees C./unit time). The unittime could be set to the period for which a particular variabledistribution of energy in the chamber relative to an item was held.Likewise, the extrapolated states and error could include numericalvalues representing an extrapolated temperature and temperaturedifference for each segment of the item. However, the error could alsobe a root mean squared (RMS) value derived from such numerical values.

The response vectors and target state vector could be utilized by asolver as part of the optimization analysis to develop a plan to heatthe item from a current state to a target state. The optimizationanalysis could select the responses, and potentially multiplerepetitions of those responses, that lead from a current state to atarget state. As illustrated in FIG. 4, and assuming an initial state ofall white cells (nominal low temperature), extrapolated state 404 wouldbe the extrapolated state expected from the application of theconditions that lead to response 401 followed by the application of theconditions that lead to response 402. This is represented by the factthat state 404 is a combination of the shading in responses 401 and 402.

The optimization analysis solver could select the responses thatminimized an error vector represented by grid 406. Minimizing the errorcan involve minimizing a difference between the target state and theextrapolated state on a segment-by-segment basis. However, the errorterm can be more complex in that overshoots on temperature can bepenalized relative to undershoots on temperature. In addition, errors onone portion of the item could be penalized more heavily than others. Inparticular, if the identity of the item has been ascertained, the errorterm can heavily penalize overheating on foods that are prone toburning, smoking, or dehydrating. In the illustrated case ofextrapolated state 404, corresponding to plan 1, the error vector 406associated with plan 1 includes values for two cells, 407 and 408, thatwere heated to a higher temperature than desired. This level of errorcould be considered acceptable, in which case plan 1 would be acceptedby the control system and executed, or it could be consideredunacceptable and lead to the execution of additional processes toproduce a more accurate plan.

Data flow diagram 500 can be utilized to describe a particularoptimization analyses that would lead to the generation of plan 1 fromFIG. 4 in accordance with the execution of step 305 from FIG. 3. Dataflow diagram 500 includes two response vectors 501 and 502 thatcorrespond to responses 401 and 402 from FIG. 4. The response vectorsdescribe how the item responds to an application of heat. Asillustrated, the response vectors include eight temperature derivativevalues each corresponding to a segment, a respective application ofenergy, and a respective configuration of the electronic oven (i.e.,dT_(xy)/dt where x is the segment number and y is the respectivecondition). Again, a respective condition is defined as by a respectiveapplication of energy delivered while the electronic oven is in arespective configuration. The response vectors can be combined toproduce a response matrix A. The response matrix, a target state vectorb_(target), and a current state vector b_(current) could be utilized inan equation 504 by a solver 505 to select a duration vector {circumflexover (x)}.

Duration vector {circumflex over (x)} includes a set of numerical valuescorresponding to a duration for each condition in a plan generated bythe convex optimization solver. For example, the duration vector couldbe a vector such as duration vector 506 and include a numberrepresenting a time for which each condition should be held. The numberscould be integers to indicate the given condition should be held forcertain multiples of a normalized time period such as 3-5 seconds. Theduration vector could be a set of durations for the electronic oven tobe in each configuration from the set of configuration for which data isavailable in the response matrix (i.e., time_(y) where y is therespective configuration for which the duration applies). The solvercould select the duration vector to minimize the error in equation 504.In the basic example of FIGS. 4 and 5, solver 505 will produce a valueof {circumflex over (x)}=[1, 1] to indicate that the plan shouldcomprise a single application of the conditions that lead to response401 and a single application of the conditions that lead to response402.

In some applications, the solver, such as solver 505, will be a convexoptimization solver. The solver can be subject to constraints beyondminimizing an error between a target state and an extrapolated state.For example, the solver could be constrained by a maximum time allowedfor a given heat task to execute, a maximum temperature variation acrossthe item or group of segments, a maximum temperature at a specific pointon the item, group of segments, or segment, and other constraints. Thesolver can be a non-negative least squares (NNLS) solver. An NNLS solverprovides certain benefits in that the solutions are only positive valuesand it would not be possible to apply a condition for a negative amountof time. In other words, and referring to the specific example of dataflow diagram 500, the numerical values of duration vector x will all bepositive. As a result, generating the sequence of commands to execute aplan from duration vector x will be straightforward and merely requirethe application of conditions corresponding to each of the responsevectors for a period of time set by a corresponding element in theduration vector. However, other solvers can be used. For example, astandard least squares solver could be used, and functionality providedon such an electronic oven could allow certain areas of the item to cooland essentially reverse the effects of a given application of energy. Inaddition, other solver's such as mixed integer linear programming,solvers for KKT optimality conditions, and solvers for Fritz-Johnconditions, and combinatorial searchers for optimality criterion such asbranch and bound could also be utilized.

One benefit exhibited by certain embodiments described with reference toFIGS. 4 and 5 is that the optimization analysis not only obtainsinformation regarding the plan, in the form of the duration vector, butalso obtains information regarding whether additional discovery stepsare required to produce a plan that meets a certain degree of accuracy,in the form of the error value. The control system of the electronicoven can be configured to determine that the error value from theoptimization analysis exceeds an acceptable value, and can trigger theacquisition of additional information for the planning process. Theadditional information could be obtained by looping back to step 301 inFIG. 3 and heating the item with an application of energy while theelectronic oven is in an additional configuration, wherein theadditional configuration is selected upon determining that the errorvalue exceeds the acceptable error value. The additional configurationcould be a physical configuration.

The additional configuration, and indeed the configurations generally,could be selected at random subject only to the constraint that it isdifferent from physical configurations for which a response has alreadybeen measured. However, the configurations could also be selected usingsome form of intelligence such as by setting the configuration to thefurthest possible position on the configuration-space from theconfigurations that were already analyzed, or by evaluating the responseof the item to determine which configuration would likely yield the mostnovel information.

To continue the description of how the optimization analyses can alsodetermine if more information is needed, the response of the item to anapplication of energy while the electronic oven is in the additionalconfiguration is sensed using a sensor to obtain sensor data thatdefines a respective response. The new response information could thenbe used to produce an update plan based on the additional information.If the new analysis produces an extrapolated state that is within anacceptable range of the target state, the plan can be executed using theduration vector, and the control system will know that enough discoveryhas been conducted. It is possible that the additional information willnot improve the error performance such that even further discovery willneed to be conducted. Also, the discovery steps can be conductedintermittently with the execution of steps derived from a prior plan orcan be conducted entirely separately until enough data has beencollected for an accurate plan to be executed.

The manner in which additional discovery can be conducted can bedescribed with reference again to FIGS. 4 and 5. Referring back to FIG.4, the error associated with grid 406 might be greater than anacceptable value. A control system could then determine this fact andwork to obtain third response 403 by heating the item again under anadditional configuration and sensing sensor data that defines thirdresponse 403. The control system could then conduct a secondoptimization analysis to produce a second plan leading to extrapolatedstate 405. The second optimization analysis could be the same as thefirst optimization analysis except the analysis would also use dataassociated with response 403 in addition to the data associated withresponses 401 and 402. As illustrated, the additional information leadsto a superior plan. Plan 2, as represented by extrapolated state 405,would be an execution of the conditions that generated second response402, and the conditions that generated third response 403. Asillustrated, extrapolated state 405 matches target state 400 such thatthe optimization analysis would generate an error value of zero. At thatpoint, the control system would know that sufficient information hadbeen obtained and would proceed with the actual execution of the secondplan. The benefit of these approaches is that the plan is obtained atthe same time the performance of the plan is quantified, which allowsthe control system to either execute the plan immediately, or quicklydetermine that more information is needed.

A specific implementation of the approach described in the previousparagraph can be described with reference again to FIG. 5. An errorvector 507, which includes a temperature value for each segment of theitem (T₁ . . . T₈), can be calculated by multiplying the response matrixA by the duration vector X, obtained from solver 505, adding the valueto the current state vector and subtracting the target state vector.Error vector 507 therefore represents the difference between the targetstate and the extrapolated state. The resulting value can be compared toan acceptable error value Error_(o) that is set by the control system.The acceptable error value can be controlled based on the expectedperformance of the electronic oven. In a first iteration of theoptimization analysis, the response matrix A only contains responsevectors 501 and 502, corresponding to responses 401 and 402 from FIG. 4.As a result, comparator 508 will determine that the optimization outputproduced an error that is too high when compared with the acceptableerror value.

Data flow diagram 500 illustrates how an additional response vector 503can be obtained for addition to response matrix A in a second iterationof the optimization analysis. Although it is possible for an additionalresponse vector to have no impact on the error vector, the additionalflexibility afforded to the solver will generally decrease the errorvalue represented by the error vector. If this updated error value islow enough, the plan can execute without obtaining additional responsedata. However, if the updated error is still too high, an additionaliteration of the illustrated loop can be performed to obtain moreresponse data and run additional optimization analyses. Ideally,response vector 503 will correspond to a response such as response 403.Response 403 is ideal because a plan that applies the conditionsassociated with responses 402 and 403 will result in an extrapolatedstate that exactly matches the target vector.

The error value and duration vector can also be used to determine if adesired target state presents an intractable problem for the electronicoven with a specified level of acceptable error. Intelligence regardingthis determination can be built into the control loop that triggersanother round of discovery (e.g., the loop back from comparator 508 inFIG. 5). The difference between an intractable problem, and one forwhich more data is needed, can be at least partly determined bycomparing a temperature of each segment in the extrapolated state of aplan with the temperature of the segments in the target state. Ifcertain segments have temperatures in excess of the target state, butthe error value has not yet receded below the acceptable error value,then the control system has an indication that the problem may beintractable. A limit on discovery of three to five additional rounds maybe tolerated once this threshold has been crossed. At that point, theoptimization analysis may indicate that an error has occurred and ceasediscovery. Alternatively, the control system can be configured to relaxthe tolerance of the solver to allow for a larger temperature variationfrom the target state.

The duration vector generated by the optimization analyses describedabove can include information regarding an absolute duration for whicheach configuration should be applied, but might not specify the order inwhich the configuration should be applied. As such, generation of theplan may include additional processing to conduct step 306 in whichcommands are compiled into a specific sequence to execute an actual planby the electronic oven. The sequence of commands can transition theelectronic oven between the set of configurations, apply theapplications of energy, and assure that the electronic oven is in eachof the physical configurations for a duration that is proportional to arespective element in the duration vector. However, the sequence ofcommands can be compiled in various ways to achieve different results.An example of different sequencing, and the resulting effect on an itemin the chamber through the course of a heating task, can be describedwith reference to FIG. 6.

FIG. 6 includes two sets of axes 600 and 601. The two sets of axes havex-axes in units of time in seconds and y-axes in units of temperature indegrees Celsius. Each axes also includes two curves that approach twotarget temperatures. The two target temperatures, 602 and 603, are thetarget temperatures for different segments in a target state for theitem. On both sets of axes, the two segments approach their targettemperatures and then level off.

The curves on axes 600 and 601 illustrate the execution of two separateplans that were generated from the same duration vector, but werecompiled into different sequences. On axes 600, the first segment andsecond segment are heated all the way to their target temperatures inseries. On axes 601, the plan was sequenced during the compiling step tominimize a maximum temperature variation across the surface of the item.Different constrains can be applied to the compiling of a sequence, andthe constraints can vary based on an identity of the item or sub-itemsin the chamber. However, minimizing a maximum temperature variation is abeneficial approach in most instances because some of the optimizationanalyses disclosed above do not take into account a decay on thetemperature of segments and may inaccurately capture the effect ofheating one segment on another segment. These two deficiencies can leadto regions of the item being either colder or hotter than expected.Though one of these deficiencies can be minimized by having theextrapolated states keep track of temperature decreases via theinclusion of a basic decay function on a segment-by-segment basis, thedeficiencies can both be minimized by assuring that the item is heatedvia a sequence of commands that promote an even distribution of heatthrough the item throughout the heating process.

As mentioned previously, the optimization analyses can be repeatedperiodically while the item is being heated towards a target state. Theperiod for repetition can be set to a fixed time or can be reliant on adetected event. For example, if a deviation detector determines that thestate of the item has strayed too far from an extrapolated state, theoptimization analysis can be conducted again. As another example, theperiod of repetition can be set based on an observed fall-off in theaccuracy of the extrapolated states, and can be adjusted during thelifetime of an electronic oven through a machine learning system thattracks the fall off-in accuracy of the extrapolated states.

The frequency at which the additional iterations of the analysis areconducted should be controlled to assure that the accuracy of the planthat is currently being executed is maintained, while at the same timeallowing the effect of a given plan to be sensed and registered by thecontrol system. Given that the execution of each action is on the orderof seconds, and the accuracy of the heuristic's prediction tends to falloff on the order of tens of seconds, the period between additionaliterations of the analyses should be greater than 3 seconds and lessthan 15 seconds. The period can be extended or shortened to a largedegree based on an identity of the item. For example, items that tend toexhibit unpredictable and widely time variant responses can be subjectedto near continuous re-planning. On the other end of the spectrum,homogenous items like cups of tea can be heated with a lower frequencyof repetition for the optimization analysis.

The response data can be updated throughout the heating process. Assuch, additional executions of the optimization analysis conductedduring a heating task can utilize response data obtained during theexecution of a previously generated plan. In other words, sensor datathat defines a response by the item to a given condition can becollected while the oven is placed in that condition as part of theexecution of a previously generated plan. Alternatively, the additionaliterations of the optimization analysis can be associated withinterruptions of the heating task in order to run additional discovery.Regardless of how the additional response data is obtained, the data canthen be used to check if the response vector that was previouslycollected is no longer accurate. If the newly obtained additional sensordata indicates that the response vector does not match with thepreviously stored response vector for the same condition, the responsevector can be updated in the library and used during later iterations ofthe optimization analysis.

In general, the optimization analysis will be able to perform with atighter tolerance if the electronic oven is able to exhibit a largernumber of configurations with variant characteristics, and a responsefor a large number of those configurations are analyzed. In other words,both the number of discovery iterations and the actual number ofconfigurations that an electronic oven can exhibit are inverselyproportional to an appropriate tolerance for the optimization analysis.For example, in approaches that utilize a set of reflective elements,benefits accrue when the set of reflective elements includes at leastthree reflective elements and the control system can generate commandsthat independently alter all three reflective elements in the set. Inaddition, if the number of segments an item is divided into increases,with all else held equal, the number of configurations generally needsto increase to hit a required tolerance level. As a baseline, if the setof segments includes at least 10 elements, then the set of physicalconfigurations should generally include at least 10 distinct physicalconfigurations.

With specific reference to an electronic oven in which theconfigurations are physical configurations set by reflective elementslike the one illustrated in FIG. 2, with the reflective elements placedon a ceiling of the chamber, roughly homogenous items placed in thechamber can be uniformly heated to within a target acceptable errorlevel of 5 degrees Celsius with a set of at least 5 distinctconfigurations and 5 segments. If the regions of interest increase to 12regions, a set of at least 10 configurations will achieve acceptableresults under similar constraints. Non-uniform heating, andnon-homogenous items have an appreciable impact on the number ofconfigurations required. For example, in the example described withreference to FIG. 7 below, which involved a heating task calling for ahighly non-uniform distribution of heat, at least 25 configurationsprovide a likelihood of hitting targets within 2 degrees Celsius RMSerror across 12 segments.

FIG. 7 includes a set of axes 700 where the x-axis is the number ofconfigurations available to the control system, and the y-axis in unitsof RMS error across all segments in units of degrees Celsius. Thecharted curve was obtained from simulations in which there were 12regions of interest with one having a requirement of being 20 degreeshigher than all other regions. The item in this case was an array ofpools of liquid stored in a microwave-transparent container. Each poolof liquid was treated as a segment for the optimization analysis. Inthis sample, the desired number of configurations was on the order of 25configurations. However, this is a demanding requirement as it issomewhat uncommon to require a portion of an item in the chamber to be20 degrees hotter than the rest. The number of configurations could, inbasic cases, be one. In the case of a fortuitous initial discovery stateand a homogenous item, the optimization analysis may determine that atarget state can be obtained after a single round of discovery and keepthe electronic oven in a single configuration for the duration of theheating task. However, if the item is not uniform, the initialconditions vary greatly, or the configurations are highly uneven interms of how heat is distributed, the number of required configurationscan substantially increase.

The plans developed using the techniques in this section can be used incombination with more complex approaches described in other sections.The optimization analyses described in this section can be used as theheuristic or extrapolation engine for the deterministic planner controlsystem described below. For example, the approach in FIG. 3 could beextended to include generating a second plan to heat the item, where thesecond plan was developed by a more computationally intensive andaccurate planning process. The second plan could be generated using adeterministic planner as described below, and the deterministic plannercould use the plan generated in step 304 as the heuristic for estimatinga future plan cost when generating the second plan. The optimizationanalyses described in this section could also be the policy for thereinforcement learning approach described below. For example, thecontrol system could automatically heat the item in the chamber towardsa target state using a reinforcement learning system where the plangenerated in step 304 was used as a policy for the reinforcementlearning system. The policy could be used to make a rough-cutdetermination of which action to take from a given node in thereinforcement learning system while the system was attempting to take agreedy step instead of exploring the feature space.

Evaluative Feedback—Reinforcement Learning Control

A set of example computer-implemented methods utilizing both evaluativefeedback and a reinforcement learning training system to heat an item ina chamber can be described with reference to flow chart 800 in FIG. 8.In step 801, energy is applied to the item with a variable distribution.The variable distribution can be caused by the standing wave pattern ofa microwave energy source applied to the chamber. The variabledistribution can also be caused by the targeted application of energy tothe item. An example variable distribution 802 is illustrated as beingapplied to item 803. Variable distribution 802 includes a local maxima804 with a relative position 805 with respect to item 803. In thisexample, the relative position has a value of zero in step 801.

In step 810, a surface temperature distribution for the item is sensedusing an infrared sensor. The infrared sensor could be an infraredcamera 811 capturing infrared radiation from item 803. The surfacetemperature distribution 812 can be sensed while the relative positionvalue remains at zero. The surface temperature distribution can at leastpartially define a state S₁. Step 810 could also involve sensing RFparameters associated with the delivery of energy to the item. In thiscase, some aspects of step 810 could be conducted simultaneously withstep 801. State S₁ may be more fully defined by a collection ofinformation regarding the item and the instantaneous condition ofcertain controlled aspects of the system. State S₁ can be a unit of datathat is an input to the action-value function of the reinforcementlearning training system. Data from the surface temperature distributionpartially defines the state in that the data can be used either alone orin combination with other information to distinguish state S₁ from atleast one other state.

In step 820, the control system will evaluate the action-value functionF(s,a) using the first state as an input to determine a second state S₂″in a set of potential second states (S₂′, S₂″) that provides a maximumpotential reward value. This step 820 could involve providing the firststate and a set of potential actions that can be taken from the firststate to the action-value function as inputs and selecting the actionwhich maximizes the magnitude of the action-value function. For example,the inputs to the action-value function could be the current state S₁and an action that changed the relative position of the item beingheated by 10 cm with respect to the variable distribution of energyapplied to the item. This movement is illustrated in FIG. 8 by action a₂and the actual alteration of the relative position 805 from a value ofzero to 10 cm as indicated by reference number 821.

In steps 830 and 840, the control system will act upon the determinationmade using the action-value function in step 820 that found S₂″ to bethe optimal state to move to. This can be achieved by action a₂ whichcan involve moving either the item, the location of the local maxima ofthe variable distribution relative to the chamber while keeping the itemstationary, or a combination of both moving the item and the localmaxima. In keeping with the example above, in step 830, the relativeposition 805 will be altered from zero to 10 cm by the control system.In step 840, energy will be applied to the item via the variabledistribution. Steps 840 and 801 can be component parts of a continuousapplication of energy to the item, but can still be conceptualized asseparate steps for purposes of understanding the operation of thesemethods. The relative movement of item 803 and variable distribution 802is illustrated as being along the surface of item 803, but it couldinvolve a movement within the volume of item 803.

In step 850, a second surface temperature distribution for the item issensed using an infrared sensor such as infrared sensor 811 from step810. The infrared sensor could be obtaining surface temperaturedistributions for the item at a faster rate than is needed by thecontrol system and storing the distributions in a buffer or disk untilspecific samples are required by the control system. In the alternative,the infrared sensor could obtain periodic distributions as needed by thecontrol system. Step 850 could also involve sensing other parametersassociated with the state of item 803 such as RF parameters like returnloss and impedance matching.

As illustrated with respect to step 850, the movement of the localmaxima will result in a more even distribution of heat 851 across theitem. However, this will not always be the case. The movement might notresult in a more even distribution of heat or might not decrease thevariance in the distribution of heat as much as expected from theevaluation of the action-value function in step 820. Regardless, in step860, a reward value will be derived using the second surface temperaturedistribution. As mentioned previously, the reward value can beproportional to a variance in the surface temperature distribution ofthe item. The derivation in the reward value in step 860 can involve theuse of numerous other factors in combination with the surfacetemperature distribution. The derivation can alternatively involve anevaluation of RF parameters associated with the heating of item 803 suchas return loss and impedance matching.

In step 870, the action-value function will be updated based on thereward value that was derived in step 860. Before any training has takenplace, the action-value function can be initialized either randomly orwith an engineered guess as to the appropriate values of the function.As such, the evaluation conducted in step 820 to determine the maximumpotential reward value for a specific action from S₁ is conducted withimperfect information. However, the measurement conducted in step 850and the derivation of reward values in step 860 can be used to updatethe action-value function so that if the same state S₁ is encountered inthe future, the control system will have better information regardingwhat action should be taken.

The control system can be configured to randomly replace the evaluationtaken in step 820 with an exploratory selection that selects a differentaction from what the action-value function would indicate as the optimalaction. In this way, the control system is able to explore the space ofpotential states and determine if a different set of actions would leadto a better result. Steps 860 and 870 can be skipped in situations wherethe exploratory step was taken. The control system can stochasticallyvary between exploratory selections and selections guided bymaximization of the action-value function (i.e., greedy selections). Theprobability of making an exploratory selection can be altered throughoutthe course of a training episode and throughout the useful life of adevice's operation.

Utilizing the information gleaned from evaluative feedback to train thecontrol system using a reinforcement learning training system providescertain benefits when applied to a control system used to heat arbitraryitems placed in a chamber. In particular, there is no need to providepredetermined training data as in supervised learning approaches. Aslong as the rewards system is configured to guide the training systemappropriately, the merit of certain courses of action can be evaluatedregardless of whether or not they have even been considered ex ante by ahuman designer. As heating items in a chamber, in particular heatingfood items in a microwave, can be guided by a set of principles that canbe generalized across a large number of food items, such a system ofrewards can be readily developed for wide applicability to potentialtraining scenarios into which the control system could be placed withoutin-depth consideration of all the myriad characteristics of the itemsthat may be placed in the chamber by a future user. Another benefit isattributable to the fact that reinforcement learning is beneficiallyapplied to situations in which the reward signal is noisy and delayed.In the case of an electronic oven, the benefit of a particular actiontaken by the electronic oven might be delayed numerous time steps fromwhen the action was taken. This is again an artifact of the time ittakes for heat to diffuse through an item being heated. However, asreinforcement learning is a time based system, rewards offered numeroustime steps in the future can be fed back to affect decisions severaltime steps in the past to address this issue.

Some of the approaches disclosed herein include a training system thatapproximates the action-value function using a neural network. Theaction-value function will beneficially include a set of values forevery potential state in which the system could reasonably find itselfin. If the states of the control systems described with reference toFIG. 8 were guided by states that were simple matrices with temperaturevalues corresponding to the coordinates on a two dimensional plane, thenumber of potential states would be considerable as they would involveeach potential temperature at each of the locations in the twodimensional plane. Given that the states can include far moreinformation regarding the condition of the system, it is easy to see howthe number of states could become unwieldy. A function approximator canbe used to reduce the number of states required for the function. Inshort, numerous states derived from the sensors and control system wouldbe mapped to a single state with similar characteristics. The functionapproximator could be a neural network, or any back propagationregression model.

The use of a function approximator for the states utilized by thecontrols system can be described with reference to a control system withstates set by the surface temperature distribution of the item. FIG. 9illustrates three surface temperature distributions 900, 901, and 902along with a control system 903. Control system 903 includes neuralnetwork 904 and access to a set of stored actions 905. The set of storedactions include all of the potential actions that the control systemcould take from any given state. Neural network 904 serves as a functionapproximator for the action-value function F(s,a). Surface temperaturedistribution 900 could correspond to a detected current state of theitem being heated by the system. Surface temperature distributions 901and 902 could correspond to stored states that are valid inputs to thefunction used by the action-value function. Neural network 904, or anyback propagation regression model based system, could take in datarepresentative of the detected state via surface temperaturedistribution 900 and a potential action from the set of stored actions905, and provide a similar potential reward value to what would beprovided by controls system 903 if distributions 901 and 902 wereapplied to controls system 903 without having to store a specific set ofvalues for state 900. Thereby, the function approximator greatly reducesthe number of specific states that the reinforcement learning trainingsystem needs to be trained for. Control system 903 can utilize neuralnetwork 904 to execute a step similar to step 820 and output a selectedaction 906 in a less resource intensive manner than one in which valuesmust be stored for each state independently.

The logic used to assist as the function approximator for the overalltraining system may require training of its own. For example, if thetraining system is a neural network, the specific weights of the networkwill need to be trained so that the neural network becomes a fairapproximation for the action-value function. The training system for theneural networks can be a back propagation regression training model. Thedata used to train the network can be the same data used to update theaction-value function itself as described above in step 870.

Some of the approaches disclosed herein include a neural networktraining system that utilizes random samples of past experiences as thetraining data. Keeping with the specific example of a reinforcementlearning training system using an action-value function that isapproximated via a neural network, the training system can store a setof experience data points as actual observations are taking place. Forexample, the experience data point could include data to represent thereward value derived in step 860, the first state used in step 820, thesecond state determined in step 850, and the action used to transferfrom the first state to the second state in step 830. These experiencedata points could then be sampled at random to provide a set of trainingdata for the neural network. The training data could be used to trainthe neural network according to approaches where loss functions areiteratively minimized according to a stochastic gradient dissentevaluation. This approach is beneficial in that the training of theneural network can harvest multiple sets of training data from the sameset of physical measurements to increase the speed at which the functionapproximator is provided.

Evaluative Feedback—Deterministic Planner Control

The control system of an electronic oven can also include adeterministic planner to generate a plan to heat an item placed in thechamber of the electronic oven. The plan can be generated based on thecharacteristics of the item and instructions provided by the user as tohow the item should be heated. The deterministic planner could select asequence of actions to heat a specific item in accordance withinstructions provided by the user. The actions in the sequence couldeach be selected from a set of actions the electronic oven and controlsystem were capable of executing. The set of actions will depend on thecharacteristics of the electronic oven. For example, an electronic ovenwith a rotating tray may include “rotate tray clockwise 5 degrees” and“rotate tray counter clockwise 5 degrees” in its set of actions while anelectronic oven with a tray that could translate laterally in twodimensions could include “move tray left 5 cm,” “move tray right 5 cm,”“move tray back 5 cm,” and “move tray forward 5 cm” as potentialactions. Generally, the actions may include altering a relative positionof a distribution of energy in the chamber with respect to an item inthe chamber and altering an intensity of the energy applied to thechamber. After generating the plan, the control system can execute theplan by performing each of the actions in the sequence of actions tothereby heat the item in the chamber.

A flow chart 1000 of a set of methods for heating an item in a chamberwith an electronic oven that utilizes a deterministic planner isillustrated in FIG. 10. Flow chart 1000 begins with step 1001 ofapplying energy to the item from an energy source. Step 1001 cangenerally be executed in accordance with the same principals as step 801from FIG. 8. The application of energy 1002 does not heat item 10003evenly and creates an uneven surface temperature distribution on theitem. This surface temperature can be sensed in optional step 1010 whichcan generally be executed in accordance with the same principals as step810 from FIG. 8. The surface temperature distribution 1012 can be sensedby infrared sensor 10011. Information gleaned from this surfacetemperature distribution can then be used by the control system invarious ways as described below.

Flow chart 1000 continues with step 1020 in which a function isevaluated to generate a first function output. A first potential actionis used to evaluate the function. The function could be a cost functionF(n) and the function output could be a plan cost calculated withrespect to node n. Node “n” can be a node that is accessed whentraversing a graph of all the potential plans that can be executed. Thegraph can be a hyper dimensional graph where movement from one node toanother is set by all of the potential actions that can be selected whengenerating a plan to traverse the graph. For example, in an electronicoven with 4 potential actions (raise heat 1 degree, lower heat 1 degree,rotate tray left, and rotate tray right) each node would be associatedwith four direct neighbors in a forward direction and a single directneighbor in a backwards direction. Each node could be fully defined byan initialized state and a sequence of actions executed from theinitialized state. The plan cost can be a cost of executing a plan up tonode “n.” However, the plan cost can also be an estimated total plancost (i.e., an expected total cost of executing a plan that includesnode “n” from start to finish). For a planning process only concernedwith the time it takes to complete a task, the cost of the plan can beas basic as the number of steps necessary to reach the end state, or thecost could be more complex as described below.

Step 1020 is illustrated by the evaluation of the function twotimes—once for node n₂′ and one for node n₂. As illustrated, thefunction is first being evaluated with a first potential action a₁ togenerate a cost for node n₂ and then is being evaluated with a secondpotential action a₂ to generate a cost for node n₂′. Actions a₁ and a₂can each be members of mutually exclusive plans. In the illustratedcase, node n₂ is associated with a lower plan cost. As such, at leastaccording to this evaluation, the best option for minimizing the cost ofexecution of the overall plan would be to execute action a₁ next insteadof a₂.

The action can be used to evaluate the function in various ways. Forexample, the state of the item could be extrapolated using anextrapolation engine with knowledge of action a₁, knowledge of how theitem responds to certain actions, and information regarding the state ofthe item associated with state n₁. Different approaches for theextrapolation engine are discussed below. However, a basic example of anextrapolation engine for purposes of initial explanation is a physicssimulator. The physics simulator could simulate the response of the itemto specific actions that could be taken by the electronic oven. Thephysics simulator could be a thermodynamics modeling tool that took thedimensions of the electronic oven, the characteristics of the energysource, and the dimensions and characteristics of the item to simulate aresponse to a given action and thereby extrapolate the state of the itemin response to the action.

Evaluating the cost function for various nodes allows the deterministicplanner to generate a plan to heat the item in a desired manner. Assuch, flow chart 1000 continues with step 1030 in which a plan isgenerated to heat the item. The plan could be generated using the outputof the function that was evaluated in step 1020. In keeping with thebasic example provided above, the output of F(n₂) was used to generatethe plan because action a₁ was selected for the sequence of actions thatcomprise the plan instead of action a₂. In this example, two outputsfrom the cost function were used to select the plan. For more complexsituations a large number of function evaluations could be used togenerate a plan. The cost function can also be evaluated in an iterativeor recursive manner such that the plan cost calculated for any givennode could be used to either select actions for the plan or to selectnodes that should be investigated further through additional functionevaluations.

In step 1040, the plan is executed by stepping through the actions inthe sequence of actions that comprise the plan. In the illustrated case,action a₁ has been selected and results in an adjustment of the relativeposition of the distribution of energy in the chamber 1002 by a distance1041 so that the surface temperature distribution on item 1003 is moreuniform. The plan can include any number of actions and can terminatewhen item 1003 appears to be heated a desired amount. Alternatively, theplan can terminate at certain periodic intervals to allow for coursecorrections in the overall heating process. In particular, the actualtemperature of an item as it is being heated will tend to deviate fromthe extrapolated state over time due to imperfections in the performanceof the extrapolation engine. Therefore, the deterministic planner can bedesigned to produce plans of a limited duration that are followed on byadditional plans that are developed as the prior plan is being executed.The actions that can be selected from and utilized during execution ofthe plan are described in more detail in a separate section below.

The cost function can include a traversed plan cost and a future plancost. In other words, the cost function can include a component thatrepresents the cost incurred in reaching a specific node, and a costthat will be incurred by continuing from that node to a desired endstate. The traversed plan cost can be calculated using an extrapolationengine as described elsewhere in this specification. The future plancost can be calculated using a heuristic as described elsewhere in thisspecification. The extrapolation engine can provide a relativelyaccurate cost value for reaching a given node based on recursivefunction evaluations and a relatively accurate estimate of the state ofan item corresponding to the node under evaluation.

The value of the traversed plan cost is referred to as “relatively”accurate as compared to the accuracy of a future plan cost derived by aheuristic. The heuristic can provide an estimate of the cost ofcontinuing from a given node to a desired end state. The heuristic willgenerally not be as computationally intensive as the extrapolationengine and it will not need to know every single action that will allowthe control system to travel from a given node to an end state. This isin contrast to the extrapolation engine which should know each of thenodes that need to be traversed to reach the current node in order toprovide the traversed plan cost. To use the analogy of a driverless carnavigating an unfamiliar city to reach a certain point, theextrapolation engine will determine an exact description of how far thedriver has moved (traversed plan cost) after a number of turns throughthe city streets (actions), while the heuristic will just take acrow-files distance measurement from the desired end location to thecurrent location (future plan cost as estimated by heuristic). In thecurrent application, the extrapolation engine could be a physicssimulator as described above while the heuristic made a roughapproximation of the future cost. The heuristic could take a delta of asample of points on a surface temperature distribution of the item to beheated against a desired ending surface temperature distribution of theitem. The heuristic could then sum all of the deltas and multiply it bya scaling factor as a rough estimate of how much more the plan will costto complete. In different approaches, either the heuristic orextrapolation engine could operate in accordance with the optimizationanalysis described above with reference to FIGS. 3-5.

In accordance with the previous discussion, evaluating the function toobtain a total plan cost associated with given node could includemultiple sub-steps. The evaluation could include estimating a futureplan cost using a heuristic and calculating a traversed plan cost usinga state or set of states derived by an extrapolation engine. The totalplan cost for a given node would then be calculated by summing thefuture plan cost and traversed plan cost.

Approaches that utilize deterministic planners can be combined withapproaches in which an identity of the item is determined by theelectronic oven. These approaches allow the electronic oven tospecifically tailor various aspects of the plan generation and executionprocess to the item. For example, the extrapolation engine, the costfunction, and the heuristic could all be modified based on the identityof the item.

Deterministic Planner—State and Cost Derivation

In approaches in which a control system of the electronic oven utilizesa deterministic planner, the control system may need to produce costvalues for specific actions and overall plans in order to generate aplan for heating the item. The cost value can be calculated based onnumerous factors related to how the item should be heated. The costvalue can be related to the time it takes the item to be heated wherelonger heating times are associated with higher costs. A simple approachin accordance with this objective would be to have each of the actionstaken by the electronic oven designed to consume a set unit of time suchas 2 seconds, and evaluate the cost of the plan be simply summing thenumber of steps taken to traverse a graph space from the origin to acurrent node under evaluation. However, the cost could also be complexand depend on more than one factor. In some situations, the state of theitem at a given node may need to be extrapolated in order to determinethe cost of a given plan. For example, an item could be heated veryrapidly to reach an average temperature through a sequence of actionsthat heated a specific point of the item to an unacceptably hightemperature which would result in burning or charring of the item atthat point. The cost function could be adapted to avoid these kinds ofsituations, and generally provide a more nuanced approach to heating theitem, if the state of the item was extrapolated as part of theevaluation of the cost function.

In approaches in which a deterministic planner using a cost function areutilized, this process may be conducted as part of evaluating the costfunction. The state of the item can be derived using an extrapolationengine. The extrapolation engine could be a thermodynamic physicssimulator that is able to simulate the effect of certain actions on theitem and produce the next state for the item automatically. Sinceelectronic ovens tend to be highly controlled environments, thesimulator could provide a recently accurate estimate of the state of anitem in response to a set of actions. The identity of the item could berelied on to obtain this information as the simulator would need to knowthe characteristics of the item in order to accurately simulate andextrapolate its state. The extrapolation engine could extrapolate thestate of the item using the identity of the item and a model of howitems having that identity respond to heat. The model could be developedspecifically for items with that identity instead of relying on ageneralized thermodynamic physics simulator. The extrapolation enginecould also operate in accordance with FIG. 11 as described immediatelybelow in that a first surface temperature distribution observed inresponse to one action could be used to extrapolate the state of theitem in response to a sequence of additional actions.

FIG. 11 is a conceptual diagram 1100 for how an extrapolation enginecould utilize a surface temperature distribution of the item toextrapolate a state of the item. The state of the item could include aplanned surface temperature distribution for the item (i.e., a surfacetemperature distribution that the deterministic planner would expect tosee after the performance of a sequence of actions). Item 1101 isillustrated with four segments 1102, 1103, 1104, and 1105. In responseto a first action a₁, such as the application of energy to the chamber,item 1101 exhibits a first surface temperature distribution in whichsegment 1102 has been heated, and segments 1103, 1104, and 1105 havenot. Action a₁ could be executed during a discovery phase of the heatingprocess and could involve the application of a specifically tailoredapplication of energy that is used to explore how item 1101 responds toheat. The discovery phase could be conducted ex ante to any attempt toactually heat the item and could involve obtaining information as to theidentity of the item and information to be utilized by the deterministicplanner. As such, the discovery phase could be conducted before any planwas generated by the control system. However, action a₁ could also justbe any application of energy or command executed by the electronic ovenduring the ordinary course of the electronic oven's operation.Regardless, information 1106 gleaned from the surface temperaturedistribution could be delivered to an extrapolation engine 1107 forpurposes of extrapolating future states of item 1101 in response tovarious sequences of actions.

The extrapolation engine could utilize information 1106 and knowledge ofthe set of potential actions that the electronic oven can choose from toextrapolate future states of item 1101 in numerous ways. For example,the extrapolation engine could assume that the heat response of the itemwill vary proportionally with the intensity of energy applied to thechamber as compared to the intensity of the energy delivered by actiona₁. The scaling factor for this proportionality could be modified basedon an identity of the item which is separately determined as mentionedelsewhere in this disclosure. As another example, the extrapolationengine could assume that the heat response of the item will translateacross the item symmetrically with an equal variation of the relativedistribution of a pattern of electromagnetic energy in the chamber. Thisspecific example is illustrated by FIG. 11 and is described in thefollowing paragraph. As another example, the extrapolation engine couldhave a basic model of radiative and convective heat loss over time froma generic item and include that loss of heat when extrapolating theeffect of a sequence of actions over time. This model could be alteredbased on an identity of the item.

In diagram 1100, extrapolation engine 1107 receives information 1106regarding the surface temperature distribution of item 1101 caused byaction a₁. The extrapolation engine then uses that information toextrapolate states 1108 and 1109 that would result from an action a₂ andan action a₃ respectively. In the illustrated case, action a₂corresponds with a leftward shift of the distribution of energy in thechamber by one segment of the item, and action a₃ corresponds with arightward shift of the distribution of energy in the chamber by onesegment of the item. The extrapolation engine assumes that the surfacetemperature distribution will effectively translate symmetrically withthe shift of the distribution of energy such that segment 1103 will beheated by action a₂ to the same degree segment 1102 was heated by actiona₁, and segment 1104 will be heated by action a₃ to the same degreesegment 1102 was heated by action a₁.

The example used in FIG. 11 is a simplified example used to illustratethe general principle of how extrapolation engine 1107 can work. Thesurface temperature distribution resulting from a₁ will in practicegenerally be more complex and less uniform than in the example of FIG.11. However, by increasing the number of actions available to the oven(e.g., by making the intensity or position variations more granular) andincreasing the computational complexity of the extrapolation engine1107, sufficient performance can be achieved even when highly complexsurface temperature distributions are utilized by the extrapolationengine. The types of actions that can be conducted by the oven arediscussed in more detail below.

As mentioned previously, the state of the item as extrapolated by theextrapolation engine could then be used to evaluate a function such asin step 1020. Specifically, the planned surface temperature distributionof the item could be used to evaluate a cost function and generate atraversed plan cost for the control system. Returning to the example ofFIG. 11, states 1108 and 1109 as extrapolated by extrapolation engine1107 could be used to determine a cost associated with actions a₂ anda₃, and a cost of the overall plans that would include those actions. Inthe case of states 1108 and 1109, the extrapolated states reveal thatthe same % of the item is heated after each step. Given the same amountof time to conduct each step, a basic cost function might find that thecost of each action was equal. However, the cost function can takenumerous other factors into account such as the fact that the areas thathave yet to be heated are more diffuse in state 1109 as compared tostate 1108 which could be factored into the cost such that action a₂ waspreferred. Furthermore, if the cost function included a future plancost, it might be determined that it will be more expensive to completethe heating pattern from node n₂″ because 4 more actions will be neededto heat the entire item as compared to only 3 more actions from staten₂′.

In some approaches, the cost associated with a given node can becalculated without deriving the state of the item, but more nuancedapproaches can utilize an extrapolated state of the item to providegreater control to the deterministic planner. In one situation, cost issimply the number of steps required to complete a plan. The costfunction would thereby be at least partially defined by a first termthat increased with a plan duration. However, even this approach couldbenefit from an extrapolated state of the item because the deterministicplanner will be able to determine with some degree of accuracy how manysteps are actually required to execute a given plan.

The cost of any given plan or action can includes a myriad of otherfactors. For example, the cost could increase if a temperaturedistribution across the item increased to an undesirable degree, and thestate extrapolation could be used to detect this occurrence. As anotherexample, the cost could spike if a certain portion of the item in anextrapolated state exceeded a set temperature. The function would thenbe at least partially defined by a term that increases when a surfacetemperature value in the planned surface temperature distributionexceeded a threshold temperature. As another example, the cost couldincrease if the entropy of a temperature distribution across the itemincreased to an undesirable degree, and could increase in proportion tothe entropy of the temperature distribution. These approaches could bebeneficial because it is generally ineffective, and can be deleterious,to sweep an application of heat back over areas that have already beenheated to a desired degree. The various factors utilized by the costfunction could each have a linear or non-linear relationship to cost.For example, in the case of a ceiling temperature for specific items,the relationship could be a non-linear increase in cost based on theassociated factor crossing a threshold value.

As described elsewhere, the cost function could be dependent upon theidentity of the item in these case. For example, if a specific itemtended to burn or char at certain heat levels, the cost function couldspike if those heat levels were detected on an extrapolated or observedstate of the item. As another example, specific items may need to beheated according to a series of phases (i.e., the item needs to bedefrosted before being cooked). The cost function could be complexenough to account for these different requirements such as by penalizingthe application of high temperature during the defrosting phased, butnot during the cooking phase.

The extrapolation engine could be implemented as hardware on the controlsystem such as via a dedicated processor and hard coded ROM. However,the extrapolation engine could also be implemented as software routinestored in firmware on or loaded as software into the control system. Theheuristic could also involve a combination of these approaches and couldreceive updates via additional software routines loaded as software orvia firmware. Finally, the extrapolation engine could be implementedpartly on the control system actually physically present on theelectronic oven and party on a server in communication with theelectronic oven. For example, data regarding the surface temperaturedistribution and identity of the item could be obtained by theelectronic oven and preprocessed locally, while the actual extrapolationof states was executed on the server.

Deterministic Planner—Heuristic

In certain approaches that utilize a cost function with a future plancost, the future plan cost can be provided by a heuristic. The heuristiccan essentially provide an estimate of the cost of reaching a desiredstate without actually considering each of the individual actions neededto reach that state. The heuristic can be less computationally intensivethan the extrapolation engine. The heuristic also does not need to knowevery action that will be taken from the current node to a desired endgoal. Instead, the heuristic can provide an estimate for the cost offinishing a plan given the current node as a starting point. Theheuristic can provide this estimate using data from the sensors of theelectronic oven. For example, the heuristic could utilize informationderived from a surface temperature distribution of the item as obtainedby an infrared sensor. However, the heuristic could also utilizedinformation derived from a surface temperature distribution produced bythe extrapolation engine. The manner by which the heuristic calculatesan output future plan cost based on its inputs could be modified basedon an identity of the item in the chamber.

If a final surface temperature distribution of an item is the final goalof a given plan, a rough approximation for the additional cost tocomplete a plan from any given node could be estimated by comparing thatfinal surface temperature distribution against a current surfacetemperature distribution. To this end, the heuristic could obtain a setof temperature values from a surface temperature distribution of theitem. The surface temperature distribution could be extrapolated usingan extrapolation engine or sensed using a sensor. The heuristic couldthen obtain a set of delta values where each delta value corresponded toa temperature value in the set of temperature values and a desiredtemperature value from the desired final surface temperaturedistribution. The delta values could then be summed in order to obtainthe estimated future plan cost. For example, the estimated future plancost could be obtained by summing the delta values and multiplying themby a proportionality constant so that the temperature difference wasappropriately scaled to the cost function. The heuristic could also takevarious aspects of the surface temperature distribution and informationfrom other sensors into account when determining the future plan cost.

The heuristic could be implemented as hardware on the control systemsuch as via a dedicated processor and instructions hard coded into ROM.However, the extrapolation engine could also be implemented as asoftware routine stored in firmware on or loaded as software into thecontrol system. The heuristic could also involve a combination of theseapproaches and could receive updates via additional software routinesloaded as software or via firmware.

Deterministic Planner—Plan Discovery

With a small enough set of potential actions and a basic heating task,it could be possible for an extrapolation engine to simulate everypossible sequence of actions that the electronic oven would take,thereby allowing for a cost function evaluation for every potential planto traverse the graph space. However, for applications involvingarbitrary items placed in the electronic oven and a large number ofpotential actions, some degree of pruning can be done to limit thenumber of nodes in the graph that are investigated. For example, theextrapolation engine could begin from an initial state and randomlyexpand out the sequences of actions until multiple paths to the endstate were detected and the path with the lowest cost from thosemultiple paths could be selected. As another example, the expansion outfrom the initial state could be biased using a heuristic estimate of afuture plan value, the calculation of the traversed plan cost at thecurrent node, or some combination of those values. As used herein,expanding out a node refers to extrapolating information for a statethat would result by taking one or more actions starting at that node.

Returning to the example in FIG. 5, the evaluation of the cost functioncould be used to select which sequence of actions to expand upon via afurther calculation. As previously described, the surface temperaturedistribution and an action are used to evaluate a function to determinea plan cost associated n₂′ then the same surface temperaturedistribution and a second action are used to evaluate the function todetermine a plan cost associated with node n₂″. The plan investigationprocess could then continue with only extrapolating out other nodes fromnode n₂′ because the plan cost calculated for node n₂′ was lower thanthe plan cost calculated for node n₂″. The actual method by which nodeswere selected could be more complicated if multiple nodes were availablehad yet to be explored and were only one action removed from nodes thathad been explored. This set of nodes could be referred to as thefrontier of the deterministic planner. For example, the discoveryprocess could rank all of the nodes on the frontier by their plan cost,discard the bottom X % and expand the top 1-X %. Alternatively, thediscovery process could randomly expand nodes, or periodically switch toan approach in which the nodes were randomly expanded.

In many cases described above, a comparison of a first plan cost to asecond plan cost would be used to determine how to expand out thevarious sequences of actions through the graph space. As such, the costfunction and the logic guiding the discovery process could share much incommon. For example, the discovery process could favor nodes that lowervariation of temperature across the item or nodes that minimize isolatedmaxima or minima in the temperature distribution. However, benefitsaccrue to approaches in which the cost function and the logic guidingthe discovery process include certain differences. In particular,introducing randomness into the logic guiding the discovery process willassure that the discovery process is not tricked into moving through apath through the graph space that appears to produce a global minimumcost while actually only being a localized minimum. Furthermore, thedegree by which the logic for the discovery process is encouraged tobranch off may change based on the identity of the item if certain itemsare more prone to localized minima in the graph space. For example, theproportion of times the discovery process is directed to expand thefrontier randomly could be set proportional to a known degree of heatresistivity for items like the one recognized in the electronic oven.

Deterministic Planner—Deviation Detector

Electronic ovens having deterministic planners can also monitor theperformance of a plan as it executes and attempt to generate a new planif too much deviation from the expected performance is detected. Thedeviation detector could also be used to approaches using theoptimization analysis described with reference to FIGS. 3-5. Themonitoring process could be conducted continuously as a plan to heat anitem is executed. The comparison could use any sensor data available tothe control system of the electronic oven and could utilize a comparisonbased on any of the extrapolated characteristics of the item through thecourse of the plan's execution.

The extrapolated characteristics of the item could be obtained duringthe plan discovery process or during the actual plan generation process.In one approach, the extrapolated states of the item corresponding toeach node in the graph space could be saved when that node was selectedfor traversal in the chosen plan. The comparison could then occur ateach step of the plan's execution to see if the actual resulting stateof the item matched the planned state as extrapolated when the plan wasoriginally being generated.

FIG. 12 is a conceptual diagram 1200 of how the performance of the plancould be monitored. Item 1201 represents an actual physical item placedin a chamber and is illustrated as having a blank surface temperaturedistribution to indicate that no heat has yet been applied to the item.Extrapolation engine 1202 can then extrapolate state 1203 for the itemduring a plan generation phase. State 1203 is a state of item 1201represented in a memory of the control system for the electronic oven.State 1203 can be associated with a specific node in the graph space anda corresponding sequence of actions in a stored plan. Item 1204represents the same actual physical item as item 1201 except that heathas been applied to the item in the chamber to produce a surfacetemperature distribution on the item. The surface temperaturedistribution of the item could be sensed by an infrared sensor 1205. Thesurface temperature distribution measured by the infrared sensor couldbe compared to the planned surface temperature distribution associatedwith state 1203 by a comparator 1207 implemented in a separate memoryspace 1206 in the control system.

The method of monitoring the plan's performance can continue withdetecting a variance during the comparison step. A variance between aplanned state for the item and an observed state for the item during theactual execution of the plan can be measured in various ways. In a basicexample, a simple delta of sampled values from two surface temperaturedistributions such as that for item 1204 and state 1203 could be takenand compared to a given threshold for what would be considered anunacceptable variance. In other cases various factors could beconsidered an unacceptable variance. For example, the appearance of alocalized and unplanned hot spot 1208 could be considered anunacceptable variance even if a majority of the surface temperaturedistribution was in keeping with the extrapolated performance of theplan. In addition, any of the factors that could be used to dramaticallypenalize the cost function during the plan generation phase can be usedto detect an unacceptable variation. Although such factors would be goodproxies for determining when a plan needed to be adjusted, a strictvariation from the plan, regardless of the occurrence of any riskconditions, is important itself because deviations from the plan couldbe indicative of more pressing underlying flaws such as amisidentification of the item in the chamber or an item that isexperiencing undetectable conditions of greater severity such as anunobservable interior that is not responding as expected.

If a variance is detected during a comparison of the planned state andan actual observed state, the control system can take several responsiveactions. First, the deterministic planner can generate a second plan toheat the item in response to detecting the variance. The control systemcould alternatively shut down the power or switch into a default heatingmode like a timed heating routine with an automatic shut off based onthe heat of the chamber. The second plan can be generated in the sameway the first plan was generated. Alternatively, the second plan can begenerated by providing information to the extrapolation engine regardingthe variance from the expected performance. The actual observedperformance, or a delta of that performance against what was expected,can then in turn be used to improve the performance of the extrapolationengine in future heating tasks. The control system could also issue analert to a user of a deviation in an expected condition and ask for theuser to exert manual control over the heating process.

The control system can also execute another planning phase periodicallywithout reference to any variance from the planned performance. Forexample, the control system could rerun the planning process every fiveminutes. The plan could then be immediately switched from the originalplan to the new plan. Alternatively, the new plan could be kept inreserve and be ready to be put into place if an unacceptable variancewas detected by the control system. This use of the plan held in reservecould be predicated on the stored states associated with the reservedplan being an acceptable match for the actual current performance of theplan.

Control and Training System

Electronic ovens in accordance with this disclosure can include controlsystems to execute the methods disclosed herein. The controls system canbe used to instantiate the evaluative feedback and reinforcementlearning systems described above. For example, the control systems canexhibit the features of control system 903 described above. The controlsystems can also be used to instantiate the optimization analysesdescribed above. The controls system can also be used to instantiate thedeterministic planner discussed above including any associatedextrapolation engine or heuristic. The control system can beinstantiated by a processor, ASIC, or embedded system core. The controlsystem can also have access to a nonvolatile memory such as flash tostore instructions for executing the methods described herein. Thecontrol system can also have access to a working memory for executingthose instructions in combination with a processor. The hardware forinstantiating the control system can be located on a printed circuitboard or other substrate housed within an electronic oven such aselectronic oven 110. The control system could also be partiallyimplemented on a server in communication with the electronic oven 110via a network. The individual blocks of control system do not need to beinstantiated on the same physical device. Individual blocks can beinstantiated by separate data storage or physical processing devices.

FIG. 13 is a data flow diagram 1300 providing an illustration of theoperation of a control system 1301 in accordance with some of theapproaches disclosed herein. In particular, control system 1301 isamenable to utilization with the evaluative feedback and reinforcementlearning approaches disclosed herein. Control system 1301 can controlelectronic oven 110 using evaluative feedback. Control system 1301 cangenerate control information, receive state information regarding thestate of electronic oven 110 or an item within electronic oven 110, andadjust the control information based on an evaluation of that stateinformation. As illustrated, control system 1301 can provide controlinformation 1302 to other components of electronic oven 110 in order toimplement specific actions. Control system 1301 can receive state data1303 from other components of electronic oven 110, such as sensors, inorder to determine the state of operation of electronic oven 110 or anitem within electronic oven 110.

Control system 1301 can utilize a reinforcement learning trainingsystem. The training system can include a stored action-value function1304 that is evaluated with a sensed state and a set of potentialactions as inputs to determine an optimal action 1305 to take as anoutput. Control system 1301 will then generate control information 1302needed to implement optimal action 1305. The action-value functionitself and the system that evaluates the function can be instantiated bya processor and memory on electronic oven 110 or can be instantiatedfully or partially on a network accessible server. The values for theaction-value function and their correlation to specific states andactions can be stored in a memory 1306. The set of potential actions canbe stored in a memory 1307. Memory 1307 and 1306 can be local memorieson electronic oven 110 or network accessible memories on a networkaccessible server. The sensed state can be derived by state derivationsystem 1308 using state data 1303. The sensed state can also be derivedusing control information 1302.

After the action defined by control information 1302 has been carriedout, control system 1301 can receive a new set of state data 1303 andderive a reward value from that state using reward derivation system1309. The reward can then be used to update the stored action-valuefunction 1304. The reward derivation system can be instantiated by aprocessor and memory on electronic oven 110 or can be instantiated fullyor partially on a network accessible server. The operation of rewardderivation system 1309 is described in more detail below.

In certain approaches, the action-value function 1304 will be a functionapproximator such as neural network or other back propagation regressionmodel which will serve as the action-value function. The control systemcan also include a training system for the function approximator. Forexample, if the training system is a neural network, the specificweights of the network will need to be trained so that the neuralnetwork becomes a fair approximation for the action-value function.These weights could then be stored in memory 1306. The training systemfor the neural networks can be a back-propagation regression trainingmodel. The data used to train the network can be the same data sensed bythe electronic oven and used by the control system to update theaction-value function.

Some of the approaches disclosed herein include a neural networktraining system that utilizes random samples of past experiences as thetraining data. In these approaches, the data that is used to updateaction-value function 1304 needs to be stored for a longer period oftime. The data can be stored in a memory or disk on electronic oven 110.However, the data can also be stored on network accessible server 1310accessible via network 1311.

The data used to train the neural network can be more expansive than thedata used to update the action-value function. In particular, the datacan comprise a set of experience data points. The experience data pointcould include data to represent the reward value derived by rewardderivation system 1309, the first state used to select the optimalaction 1305, the second state derived by state derivation system 1308,and the action 1305 which was used to transfer from the first state tothe second state. These experience data points could then be sampled atrandom to provide a set of training data for the neural network. Thetraining data could be used to train the neural network according toapproaches where loss functions are iteratively minimized according to astochastic gradient dissent evaluation. This approach is beneficial inthat the training of the neural network can harvest multiple sets oftraining data from the same set of physical measurements to increase thespeed at which the function approximator is provided.

Network accessible server 1310 could include experience data pointscollected from multiple electronic ovens which could in turn be used totrain the function approximator for multiple electronic ovens.Experience data points can be pushed up to the server from each of thenetworked electronic ovens to run the training procedure at the serverside. However, the training data could also be pushed down from theserver to individual ovens to run the training procedure locally. Thispooling of training data from each training episode conducted by thenetwork of electronic ovens could greatly increase the speed at whichthe network of electronic ovens was trained for optimal performance.

FIG. 14 is a data flow diagram 1400 providing an illustration of theoperation of a control system 1401 in accordance with some of theapproaches disclosed herein. In particular, control system 1401 isamenable to utilization with the deterministic planner approachesdisclosed herein. Control system 1401 can control electronic oven 110 bygenerating a plan and delivering commands to the electronic oven inaccordance with that plan. The performance of the plan can also bemonitored. Control system 1401 can generate control information, receivestate information regarding the state of electronic oven 110 or an itemwithin electronic oven 110, and adjust and determine the performance ofthe plan based on that state information. As illustrated, control system1401 can provide control information 1402 to other components ofelectronic oven 110 in order to implement specific actions in accordancewith a generated plan 1405. Control system 1401 can receive state data1403 from other components of electronic oven 110, such as sensors, inorder to determine the state of operation of electronic oven 110 or anitem within electronic oven 110.

Control system 1401 can utilize a deterministic planning system toproduce a plan for heating an item in the chamber. The deterministicplanning system can include a stored cost function 1404 that isevaluated using state data 1403. The system may also use anextrapolation engine 1407, and a heuristic 1406, to evaluate the costfunction 1404. Specific nodes in the cost function can be selected forevaluation, and subsequently evaluated, using the extrapolation engine1407 or heuristic 1406. The control system will generate plan 1405 basedon these evaluations, and generate control information 1402 needed toimplement plan 1405. The cost function itself and the system thatevaluates the function can be instantiated by a processor and memory onelectronic oven 110 or can be instantiated fully or partially on anetwork accessible server. The values for the cost function, theheuristic, the extrapolation engine, and the potential actions theelectronic oven can execute can be stored in a memory 1409. Memory 1409can be a local memory on electronic oven 110 or network accessiblememories on a network accessible server. The sensed state can be derivedby state derivation system 1408 using state data 1403. The sensed statecan also be derived using control information 1402.

After the action defined by control information 1402 has been carriedout, control system 1401 can receive a new set of state data 1403 andcompare the actual performance of the plan with an expected performanceof the plan using a deviation detector 1410. The deviation detector canreceive an extrapolated state from extrapolation engine 1407 and compareit to the actual state reached by the item at the point in the plancorresponding to the extrapolated state. The extrapolation engine can beimplemented as a dedicated processor, or could be firmware or softwareexecuted on the same processor used to instantiate the system thatevaluates cost function 1404. The deviation detector can be configuredto trigger another evaluation of the cost function given the additionalinformation available to the control system associated with the planhaving been partly executed.

Data needed for the operation of the control system can be stored on amemory or disk on electronic oven 110. However, the data can also bestored on network accessible server 1410 accessible via network 1411.For example, the values used to initialize the cost function,extrapolation engine, or heuristic based on an identity of an itemplaced in the electronic oven can be stored remotely and updated as thesystem obtains more data. Likewise, data collected by the electronicoven can be uploaded to server 1410 for use by other electronic ovens.In particular, instances where the deviation detector determined thatthe plan did not lead to an expected extrapolated state could be used torefine the stored values that initialize the extrapolation engine at thenetwork accessible server 1410 for use by other electronic ovens.

State and Reward Derivation

A state derivation system is utilized to obtain feedback from the systemin any of the evaluative feedback approaches described above. Forexample, the state derivation system could be state derivation system508 or 1408. A reward derivation system, such as reward derivationsystem 509, is utilized to update the action-value function in areinforcement learning approach. As used herein, the term state canrefer to the actual physical state of the item, the electronic oven, orthe overall system. However, the term can also refer to therepresentation of those states as stored in a memory. In someapproaches, the number of actual physical states is much greater thanthe number of states stored in the memory.

The process for defining a state in memory will generally involve datafrom sensors, the control system, or from a network connection. Theactual physical state of the item is sensed by the sensors describedabove with reference to FIG. 1 that obtain state sense information, suchas 1303 and 1403. The states stored in memory can be defined by dataobtained from the sensors. For example, the states can be defined bydata regarding a temperature distribution across the item, twodimensional visible light data, three dimensional visible light data,laser monitoring temperature measurements, weight data, humidity data,temperature data, particulate concentration data, return loss data,impedance matching data, applied energy data, and other parametersregarding the physical state of the item, chamber, energy source, orelectronic oven generally. The states stored in memory may also bedefined by control information.

The state used by the control system can be defined by both factorsmeasured through the use of sensors, such as state sense information1303, as well as control information, such as control information 1302.In particular, control information in the form of information about themomentum of a particular action could be used to define the state. Ingeneral, any action enforced by the control system that includes adirectional term or whose behavior with respect to time has both apositive and negative derivative could be beneficially used by thesystem to define the state.

In certain approaches, sensors would not be needed to obtain controlinformation because the information would be derivable from the commandsgenerated by the control system itself. For example, the state couldinclude a value for the angular momentum of a mode stirrer in thechamber, but the angular momentum would not need to be sensed from thechamber and could instead be passed directly from the portion of thecontrol system responsible for adjusting that angular momentum to theportion of the control system responsible for evaluating and updatingthe action-value function. This is illustrated by the connection between1308 and 1302 in FIG. 13. Since the behavior of the mode stirrer inresponse to applied power could be evaluated and well modeled by themanufacturer, this model could be built into the control system suchthat the momentum could be derived from the commands used to control themode stirrer.

The networking interface can also provide information that can be usedto determine the state of the item. For example, location data for theelectronic oven can be derived by the oven's connectivity and used toderive an altitude for the oven which could be used to define the stateof the item. Also, location based information could also serve as anexternal channel for initializing the control system in order to betteridentify certain items that are consumed with higher frequencies incertain areas or to cook items according to certain local preferences.The local preferences and geographical consumption patterns could alsobe obtained as an initial matter by the electronic ovens themselves.

In a particular example, a surface temperature distribution for item 111could be sensed using an infrared sensor with a view of the item throughopening 114. The surface temperature distribution could then be used toidentify a state in a memory. Data from multiple sensors, or data from acombination of sensors and control information, could be used incombination to identify a state. For example, both a surface temperaturedistribution and a three dimensional image of the item could be capturedusing infrared and visible light sensors with a view of the item, andboth the distribution and image could be used to identify the state. Inanother example, both a surface temperature distribution and a momentumof a movable tray holding the item as calculated using controlinformation used to control a motor for the tray could be utilized toidentify the state. As another example, both a current location ofapplied energy and an impedance matching characteristic for the appliedpower could be utilized to identify the state.

The states can be defined via a derivation procedure that takes in rawdata from the sensors and delivers the processed data to the controlsystem. This step is optional, as in certain cases the data can bedelivered directly to the control system. For example, an infraredsensor can directly deliver a matrix of values for the IR intensitysensed by each pixel in the sensor to the control system. However, theraw pixel values could also be strategically down-sampled to ease thecomputational constraints placed on the control system. The raw datacould also be processed so that the control system would receive amatrix of IR intensity values that correspond directly to the surfacearea of the item as if that surface area were flattened from a threedimensional shape into a two dimensional plane. More complex derivationprocedures could be applied to provide an optimum degree of informationto the control system with which the states could be defined.

FIG. 15 provides an illustration of a more complex derivation procedurethat can be used to process raw sensor data regarding an item in orderto define states in memory. FIG. 15 includes two sets of images 1500 and1501 that correspond to the same system at two different times whereimage 1500 corresponds to t=0 and image 1501 corresponds to t=1. In bothimages, an infrared sensor 1502 obtains a surface temperaturedistribution. Surface temperature distribution 1503 is obtained at t=0.Surface temperature distribution 1504 is obtained at t=1. Both surfacetemperature distributions correspond to the same item in the chamber. Inthis case, the item is a set of items 1505 and 1506 on a movable tray1507. As illustrated, tray 1507 has been rotated 90° from t=0 to t=1.

A reinforcement learning system could treat surface temperaturedistributions 1503 and 1504 as the state of the system without regard tothe relative movement of the items 1505 and 1506 in the chamber.However, in certain approaches it might be beneficial to alleviate thisdegree of complexity for the control system by defining the states usinginformation regarding the position of the items and the surfacetemperature distribution. This can be done in numerous ways and thefollowing examples are provided for purposes of explaining how morecomplex derivations can alleviate strain on the control system and notas a limitation on how the control system is provided with informationin all instances.

In one example, infrared sensor 1502 could also obtain visible lightimages of items 1505 and 1506. This data would serve as part of thestate sense data, such as state sense data 1303. The information couldthen be used by a state derivation system, such as state derivationsystem 1308, to map surface temperature distributions from their rawvalues to distributions for items 1505 and 1506 themselves.

As another example, the control system could provide control informationfrom which the position of tray 1507 could be derived. This informationcould serve as control information, such as control information 1302.The information could then be used to transpose the surface temperaturedistributions to cancel out the effect of the movement of the tray at astate derivation system, such as state derivation system 1308, beforethe surface temperature distributions are used to define the states forthe control system as utilized by an action-value function 1304 or acost function 1404.

Other more complex derivations are possible through the combination ofmultiple streams of data. For example, the state could include aclassifier that actually identified a particular item in the ovenagainst a library of stored items that are commonly placed in amicrowave. The state could include placeholders for multiple items thatcould be placed in the microwave and could track individualcharacteristics of each item separately.

The actual physical state of an item placed in a microwave can vary fromthe state as defined by the control system. This occurrence can bereferred to as the hidden state issue. In certain approaches, the statewill be defined from a measurement directed at the surface of the item,and the temperature of the interior of the item will only be knownindirectly. However, the interior characteristics of an item may varywhile the outward appearance remains constant so that the same surfacemeasurement will be indicative of different internal states. Inapproaches in which only the surface temperature was monitored, thiscould lead to the hidden state problem.

The reinforcement learning approach described above provides certainbenefits with respect to the hidden state problem because it can operateon reward signals that are noisy and delayed. Eventually, the internalstate of the item will be determined. In various approaches the internalstate will be determined when it is ultimately expressed on its surfacethrough the diffusion of heat, or when the item is removed from thechamber and evaluated. Regardless, the updating of the action-valuefunction for previous states provides a rapid manner for incorporatingthis information back into the control system and allows the controlsystem to recognize hidden states and explore options for alleviatingtheir effects.

Aside from being used to evaluate the action value function, theinformation obtained regarding the state of the item can also be used toderive a reward for updating the action-value function. This action canbe conducted by a reward derivation system, such as reward derivationsystem 509. Indeed, any information gleaned by the system regarding thestate of the item can be used to derive a reward. Rewards can bepositive or negative. In a specific example, a positive reward can bederived for each state based on an evenness of cook determination. Thedetermination can involve evaluating the surface temperaturedistribution for the item and evaluating the variance of temperaturevalues across the distribution. Rewards can be derived from how manypoints in the distribution compare to a sigmoid function where positiverewards are provided for low magnitude points on the sigmoid. As anotherexample, negative rewards can be provided when a visible light detectoridentifies that a spill has occurred. Large negative rewards can bederived when smoke is detected in the chamber.

In addition to data used to define the state of the item, rewards can bederived from numerous other data sources. For example, rewards can bederived from the time it takes to heat an item to a desired degree whererapid heating is associated with positive rewards. Rewards can also beprovided via user feedback after the item has been cooked. For example,a prompt could appear on the display or be transmitted by a speaker onthe device to prompt the user to report on how well the heating wasconducted. As another example, a prompt could be sent to a user's mobilephone to request a response regarding how well the item was heated. Thereward could then be derived based on the user's response.

Actions

The example electronic ovens described with reference to FIG. 1 canconduct various actions as part of the process of obtaining evaluativefeedback or as part of generating a plan to heat the item. In approachesin which the evaluative feedback is used to train a reinforcementlearning training system these actions can be inputs to the action-valuefunction of the training system. In approaches using a deterministicplanner, these actions can be the actions that make up a plan andtraverse a graph of the plan space. In approaches using the optimizationanalyses discussed above, these actions could be the applications ofenergy and changes in configuration of the electronic oven that are usedto place the item in a given condition and monitor its response.

Generally, as described above, one set of actions include the ability toalter the relative position of item from a first position value to asecond position value with respect to a local maxima created by thevariable distribution of energy delivered to chamber by an energysource. To this end, tray 118 may be rotatable around one or more axes.Tray 118 may also be linearly movable in two dimensions along the bottomof chamber 112. Tray 118 may indeed be larger than the bounds of chamber112 and pass underneath the walls of the chamber as it moves in order tomove item 111 through a greater area. Tray 118 may also be movable inthe z-direction up and down with respect to the base of chamber 112. Inthe alternative or in combination, the variable distribution of energyprovided by source 113 may be movable as will be described in moredetail below.

Other actions that can be executed by an example control system includeadjusting the characteristics of the energy provided to the chamber suchas by cycling the power applied to energy source 113 on and off oradjusting the power applied in graded steps between a maximum andminimum level. In other approaches, the frequency of the energy appliedto chamber 112 can be modified. In other approaches, additional heatsources can be applied to the chamber in combination or in thealternative to the energy applied by energy source 113. Furthermore,water or other materials can be intermittently introduced into thechamber to alter the effect of the energy introduced to chamber 112 onitem 111. As another example, a susceptor could be periodicallyintroduced to the chamber to create higher temperature reactions. Thesusceptor could be movable within the chamber and could be occasionallyplaced in close proximity to certain items to cause Maillard reactionsor other effects only achievable at high temperatures. Other actionsthat can be executed by an example control system include moving astirrer or other agitator in the chamber that is configured to adjustthe position and composition of the item during heating. The agitatorcould be placed within an item. In certain approaches the agitator willcomprise material that is transparent to microwave energy. In otherapproaches, the agitator will be the susceptor mentioned above.

Another action the control system can execute is moving the variabledistribution provided by energy source 113 relative to the chamber 112itself. This action can be achieved in numerous ways. Example of how thevariable distribution's position can be altered are provided in U.S.Provisional Patent App. Nos. 62/315,175 filed on Mar. 30, 2016,62/349,367 filed on Jun. 13, 2016, and 62/434,179 filed on Dec. 14,2016, all of which are incorporated by reference herein in theirentirety for all purposes. For example, the variable distribution can bealtered within the chamber relative to the item by adjusting thephysical configuration of a set of variable reflectance elements such asvariable reflectance element 200 in FIG. 2.

Another example of how the control system can execute the action ofmoving the variable distribution with respect to the chamber is byutilizing an array of antennas or energy sources. The individualelements in the array could be supplied with variable levels of energyinstantaneously to alter the characteristics of the delivered energy byforming waves that cohere and interfere at different points.

Another action that can be conducted by the control system involves thecombination of altering the pattern of energy distribution relative tothe chamber itself and also altering the amount of energy supplied. Bytargeting a specific location on or within the item, and monitoring RFparameters such as the return loss and impedance matching, the reactionof that portion of the item to the delivered energy can be monitored.These characteristics, combined with knowledge regarding how the itemresponds to heat, can be utilized to measure how the heating process isprogressing.

Any of the actions mentioned above can be stored by the control systemfor selection by a deterministic planner when generating a plan to heatthe item. The potential items the planner can select from can bereferred to as the action set of the electronic oven. The actions forthe deterministic planner can include various set degrees in terms oftheir individual durations or physical extents. For example, the actionsset could be defined such that each action took the same amount of timeto execute (e.g., the actions associated with rotating a tray π/8radians or increasing the intensity of the energy source by 10% could beselected for the action set because they take the same amount of time).These approaches would alleviate one constraint on the cost functionbecause the cost function would be able to calculate a total plan timeby simply adding the number of actions taken. However, the cost functioncould also be defined to account for the time it took to execute anygiven action. Alternatively, the actions could be defined so that theyhad fixed durations or intensities without reference to preserving anysort of symmetry between the various actions in the action set. Theseapproaches would also alleviate the computation complexity of thedeterministic planner as a whole because they would limit the set ofpotential actions that would need to be explored and extrapolated. Analternative approach in which the actions have varying durations orintensities is possible, and could provide certain benefits in terms ofthe flexibility afforded to the planner, but would also increase thecomplexity of the extrapolation engine.

Control System Initialization

The control system can be initialized based on an identity of the itemplaced in the chamber. The item could be identified by analyzing theresponse of the item to an application of energy using infrared data aswell as visual light sensor data obtained from a visual light sensor.The control system can be initialized based on the category of the itemmatching a specific category or could be initialized based on thespecific item. For example, the item could be identified as anon-viscous homogenous liquid, or it could be identified as a cup oftea. The control system could then be initialized based on thisidentification. The control system could include a default configurationif the item is not identified. The control system could also havedifferent configurations based on different levels of specificity as tothe characteristics of the item. For example, the control system couldhave both a cup-of-tea configuration and a non-viscous homogenous liquidconfiguration and could fall back on the more general configuration ifthe identification of the tea was not conducted properly.

FIG. 16 is a data flow diagram illustrating the initialization ofcontrol system using data from external channels. Data from externalchannels can be used to initialize any of the control systems disclosedherein including those associated with deterministic planners,reinforcement learning approaches, and the optimization analysisdiscussed above. However, for explanatory purposes, FIG. 16 illustratesthe initialization of a control system for a reinforcement learningapproach as disclosed with reference to FIG. 13.

Data from external channels can provide information for initializingcontrol system 1301 for a heating or training episode. These channelscan also be utilized to initialize any of the aspects of the controlsystem that have been described elsewhere in this document as beingconfigurable based on an identity of the item. The external channels areillustrated as a QR code on a package 1601, a voice command 1602, atouch input 1603, and network data 1604. Data from external channels caninclude data transmitted through a scanner used to read a UPC or QR codeon the packaging of an item to be heated, a keypad command entered on atraditional keypad, a command entered on the user interface of a touchscreen, a voice command entered on a microphone of the electronic oven,or a camera combined with an image recognition classifier. In general,the data channels can include any method for entering commands orinformation to electronic oven 110 from an external source. Network data1604 can include information provided from a manufacturer of electronicoven 110 or from a user of electronic oven 110 that is providing controlinformation indirectly via a local network or the Internet. Theinformation could be provided through the network via a device with thevarious input channels described above with reference to control panel119, such as a mobile phone with a touch screen, microphone, and camera.

Once the data is received, it is used by electronic oven 110 toinitialize certain aspects of control system 1301. For example, theaction-value function itself can be initialized by altering the set ofvalues and correlations stored in memory 1306. This could involvesetting the weights and overall characteristics of a neural networkfunction approximator for the action-value function stored in memory1306. The behavior of state derivation system 1308 and reward derivationsystem 1309 could also be initialized or altered by the received data.The states used by control system 1301 could themselves be altered suchas by adjusting them to independently track the state of individualcomponents of the item in the chamber. In a specific example, theexternal channel could identify the items as a combination of chickenneeding cooking and rice needing reheating. As a result, the states usedby control system 1301 and state derivation system 1308 could beinitialized to keep track of the two separate components. The data usedto represent the state could be altered to include two separate vectorsfor each component. The reward procedure could be altered to reward slowgradual heating of the chicken and light heating of the rice.

In general, any aspect of control system 1301 including thecharacteristics of the reinforcement learning training system or thetraining system used to train the function approximator could beinitialized via data from external channels. For example, thereinforcement learning training system could be initialized based ondata from an external channel such as adjusting the probability oftaking an action-value maximizing action as opposed to an exploratoryaction.

Another external channel for information could be a preprogrammedcalibration procedure used to analyze the item in the chamber. Thecalibration procedure could be the same process used in the discoveryphase of the deterministic planner and optimization analyses describedelsewhere herein. The electronic oven could be configured to rapidlyheat the item or apply water to the chamber and study the reaction ofthe item to that stimulus to obtain information that can be used toinitialize the control system. For example, the item could be heatedwith an application of electromagnetic radiation, and the change insurface temperature distribution could be analyzed across a short periodof time to determine the heat resistivity of the item. In response todetermining that the heat resistivity was high, the control system for adeterministic planner or reinforcement learning approach could beinitialized with a high probability of taking exploratory steps in orderto address potential hidden state problems with such items. Indeed, theresponse of the item to the above-mentioned stimuli could be used by anyclassification system with access to a corpus of information regardingthe responses of different materials to those stimuli in order toidentify the item for the control system. As a specific example,different foods may exhibit different cooling curves in response to anapplication of heat, and monitoring the change in temperature of theitem in the chamber over time in response to a given stimulus couldprovide enough information to enable a trained classifier to recognizethe item. In a similar example, the change in surface temperaturedistribution after receiving the application of energy could be analyzedto supply the extrapolation engine for a deterministic planner approachwith information regarding how the item was both heated and subsequentlycooled in order to accurately extrapolate the state of the item for thedeterministic planner in response to various potential actions.

Although specific examples of utilizing data from external channels wereprovided above, in most approaches, the control system, such as controlsystem 1301, can operate with very little data from external channels.Much of the data described above could in fact be discovered by, andincorporated into, the operation of control system 1301 itself. Forexample, although an external channel could identify an item as cookedchicken in need of reheating or frozen chicken that would need to bethawed and fully cooked, control system 1301 could also learn toidentify the items and carry out the proper heating procedure to bringboth items to the desired state without external input. The sameaction-value function and set of states could be utilized for both thethawing and cooking task and the reheating task. Although the number ofstates and computational complexity of an approach which did not provideexternal data would be greater, the provisioning of external data is notnecessary.

While the specification has been described in detail with respect tospecific embodiments of the invention, it will be appreciated that thoseskilled in the art, upon attaining an understanding of the foregoing,may readily conceive of alterations to, variations of, and equivalentsto these embodiments. Any of the method steps discussed above can beconducted by a processor operating with a computer-readablenon-transitory medium storing instructions for those method steps. Thecomputer-readable medium may be memory within the electronic oven or anetwork accessible memory. Although examples in the disclosure includedheating items through the application of electromagnetic energy, anyother form of heating could be used in combination or in thealternative. The term “item” should not be limited to a singlehomogenous element and should be interpreted to include any collectionof matter that is to be heated. These and other modifications andvariations to the present invention may be practiced by those skilled inthe art, without departing from the scope of the present invention,which is more particularly set forth in the appended claims.

Reference has been made in detail to embodiments of the disclosedinvention, one or more examples of which are illustrated in theaccompanying drawings. Each example was provided by way of explanationof the present technology, not as a limitation of the presenttechnology. In fact, it will be apparent to those skilled in the artthat modifications and variations can be made in the present technologywithout departing from the scope thereof. For instance, featuresillustrated or described as part of one embodiment may be used withanother embodiment to yield a still further embodiment. Thus, it isintended that the present subject matter covers all such modificationsand variations within the scope of the appended claims and theirequivalents.

What is claimed is:
 1. A computer-implemented method for heating an itemin a chamber of an electronic oven comprising: taking a first action,wherein the first action alters at least one of a relative position andan intensity of a distribution of energy in the chamber from a microwaveenergy source; sensing a first surface temperature distribution for theitem using an infrared sensor; evaluating a function to generate afunction output, wherein information derived from the first surfacetemperature distribution and at least one potential action are used toevaluate the function; and taking a second action, wherein the secondaction alters at least one of the relative position and the intensity ofthe distribution of energy in the chamber from the microwave energysource, and wherein the second action is selected from a set ofpotential second actions based on the function output.
 2. Thecomputer-implemented method of claim 1, further comprising: sensing asecond surface temperature distribution for the item using the infraredsensor; and updating the function to change the function output to asecond function output based on the second surface temperaturedistribution; wherein the function output is a reward value; and whereinthe function is an action-value function.
 3. The computer-implementedmethod of claim 1, further comprising: deriving a plan to heat the itemin the chamber using the function output, wherein the plan is a sequenceof actions; and at least partially executing the plan to thereby heatthe item in the chamber; wherein the function is a cost function;wherein the function output is a plan cost; and wherein the secondaction is in the sequence of actions.
 4. The computer-implemented methodof claim 3, further comprising: extrapolating a planned surfacetemperature distribution using the first surface temperaturedistribution and a test set of actions; wherein the planned surfacetemperature distribution is the information derived from the firstsurface temperature distribution used to evaluate the function.
 5. Thecomputer-implemented method of claim 3, further comprising: sensing asecond surface temperature distribution for the item using the infraredsensor; comparing the second surface temperature distribution to aplanned second surface temperature distribution; detecting a varianceduring the comparing step; reevaluating the function to generate asecond function output in response to detecting the variance; deriving asecond plan to heat the item in the chamber using the second functionoutput; and at least partially executing the second plan to thereby heatthe item in the chamber.
 6. A computer-implemented method for heating anitem in a chamber comprising: applying energy to the item from amicrowave energy source; evaluating a function to generate a firstfunction output, wherein a first potential action is used to evaluatethe function; generating a plan to heat the item in the chamber usingthe first function output, wherein the plan is a sequence of actions;and executing the plan to thereby heat the item in the chamber; whereinthe function is a cost function; wherein the function output is a plancost; and wherein the first potential action is one of: (i) altering arelative position of a distribution of energy in the chamber from themicrowave energy source; and (ii) altering an intensity of thedistribution of energy in the chamber from the microwave energy source.7. The computer-implemented method of claim 6, further comprising:evaluating the function to generate a second function output, wherein asecond potential action is used to evaluate the function, wherein thesecond potential action and the first potential action are each membersof mutually exclusive plans; and selecting the plan for furtherevaluation based on a comparison of the first plan cost and the secondplan cost; wherein the second function output is a second plan cost; andwherein the second function is also used to generate the plan.
 8. Thecomputer-implemented method of claim 6, wherein evaluating the functionfurther comprises: estimating a future plan cost using a heuristic;calculating a traversed plan cost; and summing the future plan cost andthe traversed plan cost.
 9. The computer-implemented method of claim 8,wherein using the heuristic comprises: obtaining a set of temperaturevalues from a surface temperature distribution of the item; obtaining aset of delta values, wherein each delta value corresponds to atemperature value in the set of temperature values and a desiredtemperature value; and summing the set of delta values to obtain theestimated future plan cost.
 10. The computer-implemented method of claim6, further comprising: sensing a first surface temperature distributionfor the item using the infrared sensor; comparing the second surfacetemperature distribution to a planned surface temperature distribution;detecting a variance during the comparing step; deriving a second planto heat the item in response to detecting the variance; and at leastpartially executing the second plan to thereby heat the item in thechamber; wherein the planned surface temperature distribution isgenerated with the plan to heat the item in the chamber.
 11. Thecomputer-implemented method of claim 6, further comprising: sensing afirst surface temperature distribution for the item using an infraredsensor; and extrapolating a planned surface temperature distributionusing the first surface temperature distribution; wherein the plannedsurface temperature distribution is also used to evaluate the functionand generate the first function output.
 12. The computer-implementedmethod of claim 11, wherein: the function is at least partially definedby a first term that increase with a plan duration; and the function isat least partially defined by a second term that increases when asurface temperature value in the planned surface temperaturedistribution exceeds a threshold temperature.
 13. Thecomputer-implemented method of claim 11, further comprising: identifyinga category associated with the item using at least one of the infraredsensor and a visual light sensor; and initializing an extrapolationengine using the category; wherein the extrapolating of the plannedsurface temperature distribution is conducted using the extrapolationengine.
 14. A non-transitory computer-readable medium storinginstructions to execute a computer-implemented method for heating anitem in a chamber, the computer-implemented method comprising: applyingenergy to the item from a microwave energy source; evaluating a functionto generate a first function output, wherein a first potential action isused to evaluate the function; generating a plan to heat the item in thechamber using the first function output, wherein the plan is a sequenceof actions; and executing the plan to thereby heat the item in thechamber; wherein the function is a cost function; wherein the functionoutput is a plan cost; and wherein the first potential action is one of:(i) altering a relative position of a distribution of energy in thechamber from the microwave energy source; and (ii) altering an intensityof the distribution of energy in the chamber from the microwave energysource.
 15. The non-transitory computer-readable medium of claim 14, thecomputer-implemented further comprising: evaluating the function togenerate a second function output, wherein a second potential action isused to evaluate the function, wherein the second potential action andthe first potential action are each members of mutually exclusive plans;and selecting the plan for further evaluation based on a comparison ofthe first plan cost and the second plan cost; wherein the secondfunction output is a second plan cost; and wherein the second functionis also used to generate the plan.
 16. The non-transitorycomputer-readable medium of claim 14, wherein evaluating the functionfurther comprises: estimating a future plan cost using a heuristic;calculating a traversed plan cost; and summing the future plan cost andthe traversed plan cost.
 17. The non-transitory computer-readable mediumof claim 16, wherein using the heuristic comprises: obtaining a set oftemperature values from a surface temperature distribution of the item;obtaining a set of delta values, wherein each delta value corresponds toa temperature value in the set of temperature values and a desiredtemperature value; and summing the set of delta values to obtain theestimated future plan cost.
 18. The non-transitory computer-readablemedium of claim 14, the computer-implemented method further comprising:sensing a first surface temperature distribution for the item using theinfrared sensor; comparing the second surface temperature distributionto a planned surface temperature distribution; detecting a varianceduring the comparing step; deriving a second plan to heat the item inresponse to detecting the variance; and at least partially executing thesecond plan to thereby heat the item in the chamber; wherein the plannedsurface temperature distribution is generated with the plan to heat theitem in the chamber.
 19. The non-transitory computer-readable medium ofclaim 14, the computer-implemented method further comprising: sensing afirst surface temperature distribution for the item using an infraredsensor; and extrapolating a planned surface temperature distributionusing the first surface temperature distribution; wherein the plannedsurface temperature distribution is also used to evaluate the functionand generate the first function output.
 20. The non-transitorycomputer-readable medium of claim 19, wherein: the function is at leastpartially defined by a first term that increase with a plan duration;and the function is at least partially defined by a second term thatincreases when a surface temperature value in the planned surfacetemperature distribution exceeds a threshold temperature.
 21. Thenon-transitory computer-readable medium of claim 19, thecomputer-implemented method further comprising: identifying a categoryassociated with the item using at least one of the infrared sensor and avisual light sensor; and initializing an extrapolation engine using thecategory; wherein the extrapolating of the planned surface temperaturedistribution is conducted using the extrapolation engine.